Importance of Image Enhancement and CDF for Fault Assessment of Photovoltaic Module Using IR Thermal Image

Abstract

Infrared thermography is the science of measuring the infrared energy emitted by an object, translating it to apparent temperature variance, and displaying the result as an infrared image. Significantly, acquiring thermal images delivers distinctive levels of temperature differences in solar panels that correspond to their health status, which is beneficial for the early detection of defects. The proposed algorithm aims to analyze the thermal solar panel images. The acquired thermal solar panel images were segmented into solar cell sizes to provide more detailed information by region or cell area instead of the entire solar panel. This paper uses both the image histogram information and its corresponding cumulative distribution function (CDF), useful for image analysis. The acquired thermal solar panel images are enhanced using grayscale, histogram equalization, and adaptive histogram equalization to represent a domain that is easier to analyze. The experimental results reveal that the extraction results of thermal images provide better histogram and CDF features. Furthermore, the proposed scheme includes the convolutional neural network (CNN) for classifying the enhanced images, which shows that a 97% accuracy of classification was achieved. The proposed scheme could promote different thermal image applications—for example, non-physical visual recognition and fault detection analysis.

1. Introduction

Renewable energy aims to be a reliable power supply, offering fuel diversification, maintaining energy security, minimizing the risk of fuel spills, and lowering the demand for imported fuels. Nowadays, there are numerous domains of application that utilize renewable energy sources, such as medical [1], driverless cars [2], and wireless wearables [3]. As a renewable energy source, solar energy produces solar power, converted into electricity—i.e., solar cells, also called photovoltaic cells that convert sunlight into electricity [4,5,6]. The authors of [7,8] suggest that photovoltaic (PV) gets its name from the process of converting light (photons) to electricity (voltage), through a process called the photovoltaic effect. Solar panel technology boasts many advantages and is a cost-effective alternative to or extension of power sources for both urban and rural applications [9,10,11]. As described above, PVs are based on photo-radiation absorption and conversion in a range of wavelengths. Heat (i.e., infrared) is emitted from the PV panels during power conversion. This heat dissipation process and the heat emitted by an object, or a person can be captured and recorded using a thermal camera [12,13]—the use of which is increasing in different monitoring applications in fields such as engineering [14,15], structural systems [16,17], security [18,19], and medical fields [20,21]. Moreover, a thermal camera can help to highlight hazardous areas radiating temperatures beyond their normal operating limits, the overheating on an electrical system, and related abnormalities in machines. As shown in Figure 1, the corresponding thermal image of an actual gas pipe taken using our thermal camera shows the temperature (°C) variations between the gas pipe and its surrounding environment.

If thermal imaging techniques are to be utilized in PVs, the early detection of defects would be manageable. Images acquired using an infrared camera are transformed into visible images by assigning a variant of color to each infrared energy level and showing its corresponding temperature differences. The need to diagnose a photovoltaic system is necessary, which can provide a monitoring mechanism to determine the health condition of the solar panel module. The IR camera can present an even heat distribution for a healthy solar panel, while uneven heat load for a degrading one because of uneven temperature distribution.

The main objective of this study is to examine the importance of a thermal camera for fault diagnosis, which can correlate to the solar panel’s thermal signature in assessing the health conditions of the solar panels. The acquired thermal solar panel images were processed using the developed algorithm to provide detailed information on the solar panel’s health status, which is helpful for solar panel assessment. Specifically, to highlight the cumulative distributive function (CDF) as one important parameter when dealing with image processing for assessing/evaluating the health condition of solar cells/panels. In addition, a cross-validation scheme uses a convolutional neural network (CNN) method for automatic classification of whether the image to be tested is on a defective category or not. The rest of this paper is organized as follows: Section 2 highlights how thermal imaging technology is used in photovoltaic systems. Section 3 describes the proposed algorithm. Simulations, testing, and evaluation results are presented in Section 4. Finally, Section 5 concludes this study.

2. Review of Related Works

This section presents articles and research works related to the extracted features of infrared thermography in the solar panel industry.

2.1. Common Solar Panel Defects

Any imperfections in solar cells, such as cracks, poorly soldered joints, and mismatches, lead to increased resistance which eventually becomes hot spots. The long-term effects of hot spots include burn marks that degrade solar cells and the PV backsheets, which may eventually lead to fires if left unchecked. The most common solar panel defect is that of broken or chipped solar cells. This problem is typically a sign of poor power production, which reduces the energy production of a solar module [22,23]. A low-quality manufacturing system may accidentally incorporate cracked solar cells into solar panels, introducing a mismatch to the cells, which affects power production. Moreover, scratches are solar panel defects; they may hold dirt and moisture, which can penetrate and cause damage to the cells. The scratches in the encapsulation seem to have a minimal effect on the solar cells, but they minimize the power production of the solar panel [24]. A hot spot occurs in the solar cell when a number of series connected cells causes a large reverse bias across the shaded cell, resulting in substantial power dissipation. The enormous power dissipation in a small area contributes to localized excessive heating or “hot spots,” which causes damaging effects, for example, cell or glass cracking, solder melting, or solar cell degradation [25]. In addition, microcracks also have the potential to produce hot spots. Microcracks occur when the internal resistance of the damaged cell rises, causing an increase in the cell temperature as the current passes through [26,27].

2.2. Infrared Thermography

A thermal image is a particular type of visual information that represents thermal radiation into the visible range perceived by the human eye [28]. Thermal cameras and other infrared measurement instruments capture infrared radiation emitted by an object and convert it into an electronic signal which indicates temperature difference on the output thermal image [29,30]. Several research works that have been published on the uses of infrared thermography for the field of photovoltaic systems follow below. In previous research presented in [31], infrared thermography was utilized for the detection of solar panel defects in terms of panel assembly and arrangement; again, the scarcity of the dataset used was one of the drawbacks of this work because of the limited period in which the dataset was acquired—in this work, the number of samples in the dataset was too small, which affected the results reliability—and the authors did not disclose even the basic information of the measured solar cell modules.

In [32] the authors used a machine learning technique based on the extracted features of thermal images to classify defective and non-defective solar panels. However, the main limitation in the proposed approach was the insufficient number of available datasets, which affected the model’s reliability. A previous study [33] presented a scheme for the image processing technique of thermal images for the automatic identification of solar modules that occur in a hot spot, and the experimental results exhibited 97% accuracy in the visual evaluation. Having a finite quantity of datasets does not make a model accurate. The author should have used publicly available datasets that contained sufficient images to make their results more reliable. Moreover, the research only used a bounded dataset. The authors of [34] implemented a similar set-up for aerial thermal image acquisition. The reliability of the model was not entirely dependable with a confined set. It suffered when an open-source dataset was used for verification.

In the proposed scheme, the dataset images were also taken over 2 years to provide multiple variants of the thermal images. Using thermal images to detect even small temperature changes can provide insight into the health of a solar panel. Moreover, utilizing thermal imaging to inspect solar panels does not require a shutdown of the system. Finally, in this paper, various image processing techniques are implemented to enhance the aforementioned thermal images to provide an easy-to-understand domain for fault assessment. The significance of the cumulative distributive function in image processing was utilized for analysis.

2.3. Cumulative Distribution Function (CDF)

Typically, a histogram is more intuitive when dealing with image processing assessment; however, cumulative distribution function (CDF) offers a more beneficial concept. Figure 2b shows the histogram of the thermal image in Figure 2a. It shows that, with the help of a histogram, this image is represented in a domain of numbers. The resulting number set is divided into a number of uniform intervals—bins [35,36]. The absolute or relative number count within each bin is then plotted as a bar at the appropriate intervals. The CDF, on the other hand, plots the percentage or relative count of the sorted numbers over the numbers themselves [37], as shown in Figure 2c. This graph demonstrates that a significant number of F(x) values from the given set of numbers are less than or equal to x, demonstrating a number of essential benefits.

Figure 3 shows the fundamental values and features such as minimum, maximum, median, which are the primary benefits of the CDF over the histogram. The minimum can be seen right where the CDF starts and intersects the x-axis. The limit can be seen where the CDF intersects the line y ends. Each value in the given set of numbers corresponds to a specific point in the CDF. The number of samples in the histogram cannot be addressed individually.

3. Methodology of the Proposed Concept

The descriptions of each stage of the proposed work are mentioned below. Pre-image processing was implemented to improve the raw image. Then, all images were subjected to various image processing techniques for enhancement. The image extraction algorithm was implemented on the thermal images. Lastly, a convolutional neural network technique was added for automatic image classification to determine the accuracy of classification on the testing image.

3.1. Data Acquisition

The dataset was acquired in our testbed, the duration of data gathering was from July 2019 to the present. Thermal images were taken every three hours from 6:00 AM to 6:00 PM. The experimental set-up is shown in Figure 4. All acquired thermal solar images were segmented into solar cell sizes, as shown in Figure 5, to provide more detailed information by region or cell area rather than the entire solar panel. This way, the feature extraction approach will provide a more significant set of solar panel features and characteristics.

Table 1. Thermal camera technical specifications.

Brand/Company Name	FLIR E8
Field of view (FOV)	45° × 34°
Object temperature range	−20 to 250 °C
Image frequency	9 Hz
Thermal sensitivity	<0.06 °C
Accuracy	± 2 °C
Thermal palettes	Iron, Rainbow, Greyscale
File format	Radiometric JPG
On-board digital camera	640 × 480

shows the technical specifications of the thermal camera used in this experiment. The technical specifications of the solar panel are shown in

Table 2. Solar panels technical specifications.

PV Module Brand/Company Name	LUMOS SOLAR (LLC)
Cell technology	Monocrystalline
Maximum power (PMAX)	270 W
Short-circuit current (ISC)	9.74 A
Open-circuit voltage (VOC)	27.78 V
Current of maximum power (IPMAX)	9.08 A
Voltage of maximum power (VPMAX)	33.08 V
Module area (Size)	1645 by 983 by 42 (mm3)

. The proposed block of the algorithm is shown in Figure 6.

3.2. Pre-Image Processing

Pre-image processing operations aim to improve the raw image data that are suppressed by unwanted distortions on some image features, which is essential for further processing like feature extraction [38]. The pre-image processing is composed of size adjustment and resolution enhancement. The pre-processing technique (e.g., resizing and filtering or denoising) aimed to improve the raw image data suppressed by unwanted distortions of some image features. Adjustments in size were made to ensure uniformity and the proper execution of statistical measurements.

Table 3. Image details/information of all thermal solar cell images.

Attribute	Value
Width (columns)	400
Height (rows)	400
Class	uint16
Image type	False color

provides the information of the thermal solar cell images.

The pre-processing technique, as demonstrated in Algorithm 1, made use of the MATLAB functions. These functions such as imread, imshow, imresize, medfilt2 were carried out using the MATLAB 2020b platform, and a brief discussion of each step is included in the following section. These syntaxes can be found in the MATLAB Documentation (https://www.mathworks.com/help/matlab/ (accessed on 7 January 2021)). In addition, the filtering procedure is defined in step 3 of the procedure.

Algorithm 1. Pre-processing layer

The steps are as follows: Step 1: Loading and Reading of Images. In this step, the raw thermal solar cell images are to import the libraries of the samples to use for this proposal. First, the MATLAB function “dir” prepares the listing of the files and folders in the current folder for the task. Then, the “imread” function was implemented at this stage. Below are the description syntaxes of these two functions. A = imread(filename, fmt); %this function reads the image from the file specified by filename and saved in the variable “A” dir name; %provides the lists files and folders in the current folder Step 2: Resizing of Images. In order to visualize the change in the images, the two functions were created; the first is to display the sample (original) image and second is to compare the original and resulting image. In this study, the base sizes of all sample images are set into a 400 × 400 pixel value. The MATLAB function “imresize” provides the image in the scale size necessary for the required task. The function “imshow” displays the image for visual verification. Step 3. Image Filtering. In this study, median filtering (Pp) was implemented to remove noise preserving the edges of the sample images. The noise reduction aims to improve the results or the later processing of this study. The median filter operates by moving the image pixel by pixel and replacing each value with neighboring pixel’s median value. The pattern of neighbors is known as the “window,” and it slides pixel by pixel across the entire image, 2 pixel by pixel. The median is calculated by first sorting all of the pixel values in the window into numerical order and then replacing the pixel under consideration with the pixel with the middle (median) value. Equation (1) moved to the initial dataset (DO) to produce the pre-processed dataset (PD). PD = DO * Pp (1)   Note: The operator “*” in Equation (1) performs the MATLAB function of median filtering on the said image dataset.

The P_P collects the pixel values under the filter and takes the median of those values; the result is then assigned to the center pixel. Then, the median is the number that lies in the center of a sorted set of numbers. Additionally, the numbers are sorted from lowest to highest to get the median, and the number in the middle will be our median value. The MATLAB function “medfilt2 (I)” was used to perform the median filtering of the image I in two dimensions which were used in this study.

3.3. Grayscale Image

A grayscale image represents an equal intensity of red, green, and blue components in the RGB space [39]. For this study, the initial feature extraction step is to transform the input thermal image to grayscale, which immediately improves the image’s appearance. The objective is to deepen the variation of contrast at a pixel value. The authors of [40] showed that grayscale imaging aided in distinctly measuring a region of an image using pixel values. The more significant the value, the greater the difference in its contrast.

Image enhancement techniques aim to provide better input and improve the interpretability of information in images for better analysis and assessment. Since the acquired thermal image is in RGB, first, it was converted into grayscale (GS). The mean value of the three colors of the RGB image is obtained using Equation (2):

$$GS=\frac{\left(R+G+B\right)}{3}$$

(2)

3.4. Equalization of a Histogram

The histogram equalization in this paper allows these areas to display a higher contrast. The histogram accomplishes this by effectively distributing the most frequent intensity values. The histogram shows a graphical interpretation of the tonal intensity distributed in a digital image [41]. Usually, the distribution of its discrete intensity levels is within the range of 0 to L − 1, where L is the total number of gray levels in the image (typically 256). The tonal intensity distribution is a discrete function h that is associated with each intensity level: the number of pixels (r_k) with this intensity (n_k). Histogram equalization is a method to improve the contrast in images by modifying the intensity distribution of the histogram. In [42] the authors demonstrated the enhancement of an image that improved its visual effect and overall brightness and clarity. The objective of histogram equalization is to provide linearity to an image’s cumulative distribution function (CDF). As shown in Equation (3), the CDF of the histogram is defined as the cumulative sum of all the probabilities within its domain [34,43]:

CDF (x) = P (X ≤ x)

(3)

Equation (2) shows a value less than or equal to x of the random variable X (e.g., image), where n_kn, is the variation in the intensity level of the image. Figure 7 shows the effect of histogram equalization.

Histogram equalization is an essential technique for enhancing low-contrast images. In this work, the ‘histogram equalization’ function is utilized on the thermal solar cell images. The equalized histogram image q is defined by:

$$q_{\mathrm{i},\mathrm{j}}=floor \left(\mathrm{L}-1\right){\displaystyle \sum }_{\mathrm{n}=0}^{f_{i,j}}{p}_n$$

(4)

At the normalized histogram of f with a bin given below:

$$p_n=\frac{number of pixels with intensity n}{total number of pixels} n=0, 1,\dots \mathrm{L}-1$$

(5)

where floor() is rounded off to the nearest integer, which is equivalent to transforming the pixel intensities, k, of f using the function below:

$$\mathrm{HE}\left(k\right)=floor (\left(\mathrm{L}-1\right){\displaystyle \sum }_{\mathrm{n}=0}^k{p}_n)$$

(6)

3.5. Adaptive Histogram Equalization

In this study, adaptive histogram equalization (AHE) was utilized to increase the local contrast and increase the aspects of edges in each section of a thermal image. This method splits the target image into tiles, and the histogram of each tile is used to generate the intensity remapping function for the respective tile [44]. This intensity remapping performs bilinear interpolation to smooth the inter-tile boundaries, because the basis of contrast enhancement of AHE depends on small regions, based on their local CDF [45]. Moreover, AHE enhances the local contrast and definitions of edges in each region of an image [46]. The AHE enhances the contrast of the images, which aims to provide a linear trend in the cumulative probability function associated with the image. By comparison, the histogram equalization improves on the entire image, while AHE operates on small regions in an image, called tiles. AHE enhances the contrast of each tile and redistributes the lightness values of the image.

If the f_i gray value of the ith gray level in the original image, the position j of the mapped gray level g_j is given by the ratio of ${{\displaystyle \sum }}_{k=0}^{i-1}{p}_k$ and ${{\displaystyle \sum }}_{k=i+1}^{m-1}{p}_k$ where m is the number of gray levels in the original image. Then, i and j are compared to achieve a uniform distribution or local uniform distribution; if j < i, then the gray color mapping proceeds; otherwise, (j > i), then the gray mapping recedes.

$$j=\left(m-1\right)\frac{{{\displaystyle \sum }}_{k=0}^{i-1}{p}_k}{{{\displaystyle \sum }}_{k=0}^{i=1}{p}_k+{{\displaystyle \sum }}_{k=i+1}^{m-1}{p}_k}$$

(7)

When it comes to image segmentation, gray-level thresholding is a technique that is both efficient and accurate. It is a simple and widely used method that is both efficient and accurate. Furthermore, it is quick and easy to use. Consequently, because the quality of gray-level thresholding of an image is such a critical issue, choosing the appropriate gray-level threshold variance is critical. It is possible to successfully remove the unnecessary segments of an image using the automatic gray-level thresholding technique described in this paper.

Note: Basically, a good threshold can be selected if the provided histogram characteristics are tall, narrow, symmetric, and separated by deep valleys. Additionally, the thermal image is in uint8. Therefore, it was transformed into uint16 using the function “im2uint16”.

The Algorithm 2 implemented the data mining layer, as specified above. The MATLAB functions were used to carry out the procedures. Grayscaling is accomplished using the “rgb2gray” function, histogram equalization is implemented using the “histeq” function, and adaptive histogram equalization used the “adapthisteq” function. Finally, the gray-level thresholding was performed using the “graythresh” function. Each step of Algorithm 2 is briefly described. These syntaxes are documented in the MATLAB Documentation (https://help.mathworks.com/matlab/ (accessed on 7 January 2021)).

Algorithm 2. Data mining layer

The steps are as follows: Step 1: Load the pre-processed dataset from Algorithm 1 and assign it as PD. Step 2: Grayscaling of image. In this study, grayscale (GS) representations are frequently used for extracting descriptors rather than directly operating on color images because grayscale simplifies the algorithm and reduces computational requirements. The MATLAB function “rgb2gray” was utilized to convert the sample image to a grayscale image, which removes the hue and saturation content information while retaining the luminance. Equation (8) provided the grayscale implementation (GSI). GSI = (PD * GS) (8)   Note: The operator “*” in Equation (8) shows that the image dataset is subjected to the MATLAB function of grayscaling. Step 3: Histogram Equalization. The histogram equalization (HE) enables areas with low local contrast to gain a boost in contrast. Equalization of the histogram accomplishes this by effectively spreading out the densely populated intensity values that previously weakened image contrast. The “histeq” function of MATLAB is used to transform the grayscale image. As displayed in Equation (9), the implemented histogram equalization on the GSI. HEI = (GSI * HE) (9) where HEI is the output image of the histogram equalization.   Note: The operator “*” in Equation (9) performs the MATLAB function of applying the histogram equalization on the said image dataset. Step 4: Adaptive Histogram Equalization. In this paper, the adaptive histogram equalization technique is used to improve the contrast in images by computing multiple histograms, each corresponding to a different section of the image, and using them to redistribute the image’s lightness values. As a result, it is well suited for increasing local contrast and sharpening the edges in each region of an image. The “adapthisteq” function operates on small regions in the image, called tiles. This function increases the contrast of each tile so that the output region’s histogram approximates a specified histogram. Equation (10) illustrates the adaptive histogram equalization. AHEI = (HEI * AHE) (10) where AHEI is the implemented output image of the adaptive histogram equalization.   Note: The operator “*” in Equation (10) performs the MATLAB function of adaptive histogram equalization which is implemented on the said image dataset. Step 5: Gray-level thresholding. In this study, the quality of gray-level thresholding of an image is a critical issue, therefore, the selection of a right grey-level threshold variance is important. In this paper, the global gray-level thresholding manages to remove the unnecessary segments of the image. The function “graythresh (I)” of MATLAB computes a global threshold T of the image I, using Otsu’s method. Otsu’s method selects a threshold that minimizes the intraclass variance of the black and white pixels that have been thresholded. With imbinarize, the global threshold T can be used to convert a grayscale image to a binary image. Equation (11) displays the implemented G=gray-level threshold (GLT). GLTD = (AHEI * GLT) (11) where GLTD is the output image of the gray-level threshold.   Note: The operator “*” in Equation (11) performs the MATLAB function of gray-level thresholding.

3.6. Image Classifier

Then, after the feature extraction and image enhancement, a non-defective/defective solar panel cell detection method is used to classify images using CNN as a feature extractor, and SVM acts as a classifier. In the architecture presented below, the output of the completely connected layer provides the input feature vectors for the SVM classifier in the proposed scheme. An SVM classifier, specifically the multiclass error-correcting codes model, was used as the final stage of the proposed defect detection. The SVM classifier was used in place of the CNN’s softmax layer. SVM’s objective is to determine the best hyperplane f(w,x) = w.x + b for separating two classes in a provided dataset, using the features x ∈ R^m.

The parameter w is learned by SVM by solving an optimization problem, as shown in Equation (12)

$$min\frac{1}{p}{w}^{T}w+C{\displaystyle \sum }_{i=1}^{p}max \left(0, 1-{y^{\prime }}_i\left(w^T{x}_i+b\right)\right)$$

(12)

where w^Tw is the L1 norm; while C is the penalty parameter (which can be an arbitrary value or one chosen by hyper-parameter tuning), y′ is the actual label; and the w^Tx + b is stands for the predictor function.

Equation (12) which is also known as the L1-SVM, and it uses the typical hinge loss. The differentiable equivalent is the L2-SVM which gives more steady results.

$$min\frac{1}{p}{w}^T{\Vert w\Vert }_2^2+C{\displaystyle \sum }_{i=1}^{p}max {\left(0, 1-{y^{\prime }}_i\left(w^T{x}_i+b\right)\right)}^2$$

(13)

where ${\Vert w\Vert }_2$ denotes the Euclidean norm or the L2 norm with the squared hinge loss.

Since the radial basis function (RBF) was implemented as the kernel in SVM, cross-validation was used to obtain the optimal kernel values. Moreover, the experimental set-up was simulated via MATLAB platform, both the ground truth/actual and predicted label; the positive is assigned for the defective, while negative for a non-defective input image

As shown in

Table 4. Image distribution of dataset.

Dataset	Histogram Equalization	Adaptive Histogram Equalization
Training	1890	1890
Testing	810	810
Total	2700	2700

, for the image dataset distribution, 70% were used for training and 30% for testing.

The HE and AHE training data consist of pairs of training samples {(x₁, y₁), … (x_i, y_i)}, where x_i is the observation or input feature for the ith sample and y_i ∈ {1, 0} is the associated class label. The SVM classifier is the discriminant function that maps an input feature space x_i into a class mark y_i.

The results from this module part can then be used to localize defects. The maximum epoch for the training is 100. The learning rate applied is 1 × 10⁻⁴, and the validation frequency is set to 1000. The full description is as follows for this module part of the classification. The applied convolutional neural network (CNN) is useful as a feature extractor and a classifier [47,48] in a wide variety of real-world applications. The Keras sequential model was used in our proposed work, and it consisted of convolutional, activation, and max-pooling layers. Below, the proposed implemented base CNN is presented with the following architectural details

(1)

INPUT: 32 × 32 × 1

(2)

CONV1: 3 × 3 size, 32 filters, 1 stride

(3)

ReLu: max (0, h_θ(x))

(4)

POOL: 2 × 2 size, 1 stride

(5)

CONV2: 3 × 3 size, 64 filters, 1 stride

(6)

ReLU: max (0, h_θ(x))

(7)

POOL: 2 × 2 size, 1 stride

(8)

Fully Connected (FC): 1024 Hidden Neurons

(9)

DROPOUT: p = 0.5

(10)

FC: 2 output classes

At the 10th layer of the CNN, instead of the conventional softmax function with the cross entropy function (for computing loss), the L2-SVM is implemented. That is, the output shall be translated to the following case y ∈ {−1, +1}, and the loss is computed by Equation (8). The weight parameters are then learned using [49].

Finally, to introduce non-linearities into the algorithm, an activation function is used. In the absence of this, the model will learn only linear mappings. The ReLU function is the most often utilized activation function [50]. ReLU is frequently used instead of tanh and sigmoid because it significantly accelerates stochastic gradient descent convergence when compared to the other two functions.

Furthermore, unlike tanh and sigmoid, which need substantial computation, ReLU is achieved by simply thresholding matrix values at zero (see Equation (14)).

f (h_θ(x)) = h(x)⁺ = max(0, h_θ(x))

(14)

As shown in Figure 8, the first convolutional layer consists of 32 filters with a 3 × 3 pixel dimension. After filtering, a ratio of 2 was used to enable the max-pooling process.

The primary function of the convolutional layers is to detect the local connections of features from the preceding layer. The output result of the feature maps is then sent to the activation layer ReLU. The max-pooling procedure was used in vision systems for two reasons: (1) to exclude non-maximal values, which reduces the calculation time for layers; and (2) to perform down-sampling operations on 2x2 subregions to reduce the dimension of the intermediate feature vectors. Then, filters are stacked together, and fully connected layers can be used to compute the class scores. In the proposed system, the output from the fully connected layer becomes the input feature vectors for the SVM classifier.

Table 5. Hyper-parameters used for CNN and CNN-SVM models.

Hyper-Parameters	CNN	CNN-SVM
Batch size	128	128
Dropout rate	0.5	0.5
Learning rate	1 × 10−3	1 ×10−3
Steps	10,000	10,000
SVMC	N/A	1

presents the hyper-parameters which were manually assigned and implemented in our experiments. Figure 9 shows the legend of the confusion matrix for the classification.

Below are the following equations for the confusion matrix interpretation.

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(15)

$$Precision=\frac{TP}{TP+FP}$$

(16)

$$Sensitivity=\frac{TP}{TP+FN}$$

(17)

$$Specificity=\frac{TN}{TN+FP}$$

(18)

$$Negative Predictive Value=\frac{TN}{TN+FN}$$

(19)

4. Results and Discussion

Figure 10 shows images taken from the actual testbed using an ordinary camera and a thermal camera. The image histogram was plotted using the pixel values vs. the number of pixels. The histogram of the solar cell image taken from the ordinary camera exhibits a good bimodal distribution. One peak represents the object pixels, while the other peak represents the background. This is good for selecting the threshold of an image. Conversely, the histogram of the thermal solar cell image shows a good distribution of pixels over the entire intensity range. The histogram also shows most pixel values clustered in a small area, whereas the top half of the intensity values are occupied by only a few pixels. The more the pixels are evenly distributed over the entire intensity range, the easier the image can be transformed. Thus, a flat output is obtained.

The CDF is more relatively linear in the thermal solar cell image. Therefore, the extracted features can provide more convenient and feasible health monitoring and assessment of solar panels.

Figure 11 shows the histogram equalization and AHE of the sample solar cell images. It can be seen that the image produced using histogram equalization contains a region that is significantly lighter or darker, the contrast of which is not sufficiently enhanced throughout the image. AHE improves the image by converting each pixel with a transformed function from a neighborhood region.

4.1. Histogram Comparison

Figure 12 shows the histogram equalization of the sample images, which exhibit widespread values of pixel intensity. The image histogram from the ordinary camera does not provide a better histogram than of the image taken from the thermal camera. The possible implication is that any noise in the image taken from the ordinary camera may also be enhanced during the equalization process. AHE is used to improve the contrast of the image.

4.2. CDF Comparison

Figure 13 shows the histogram equalization of both sample images. It can be seen that the grayscale was normalized. Linear normalization by getting the mean was used in this study. As a result, the thermal image provides a more linear CDF, and its range was extended to occupy the entire spectrum of levels.

Figure 14 shows a portion of three different solar cell images under three different conditions. All three images were simulated to determine the difference in their histogram domain.

As shown in Figure 15, the three histogram results of the image from the ordinary camera show a non-bimodal scenario compared with the thermal camera on the panel. The histogram of the non-defective image (thermal image—panel 1) indicates both the histogram equalization and AHE exhibit a better-distributed pixel intensity value than the defective thermal image (panel 2).

4.3. CDF Comparison

Lastly, as shown in Figure 16, the CDF comparison indicates that the thermal imaging technique provided a more linear CDF compared to the sample image taken from the ordinary camera. Moreover, the CDF under the grayscale of both defective solar panels is not similar to the non-defective sample.

Moreover, just to validate the proposed work, electroluminescence (EL) imaging technique was applied on the solar panel testbed to identify the current health status of the solar panels used in the experimental set-up. Electroluminescence is based on the same concept as light-emitting diode (LED). Normally, a current is fed using a large diode into a solar cell, and the radiative recombination of carriers causes the light emission. As an indirect bandgap semiconductor, most recombination in silicon happens through defects or Auger recombination [51]. While electroluminescence (EL) imaging is a measurement technique used to analyze solar cells and solar modules [52] that requires electrical connections, this means it is only applicable to solar cells and modules in operation [22]. The main advantage of EL imaging in PV applications is detecting micro-cracks and cracks on crystalline silicon solar cells that are invisible to the naked eye. The EL image of the testbed is shown in Figure 17. Moreover, the EL image profile of the solar panels used in the experiment is shown in Figure 18. Figure 19 shows the location of the cell defects on panel 2 for both the thermal and electroluminescence images

Figure 20 shows the different simulated images with the non-defective solar cells with the defective solar cell images. It demonstrates that the irregularities in the results on all defective images, both grayscale, histogram equalization, and adaptive histogram equalization modes are not relatively linear compared to the non-defective solar cell image.

It shows that the CDF of the sample in the simulated experiments can be validated using the EL images in determining the health status of the solar panel/cell. Although EL is good at detecting defects, it is very costly to set up, which is mostly used in the laboratory.

The results of the experiment were then compared between the CNN method and the proposed method, which includes CNN and SVM. At 10,000 steps, Figure 21 shows that the AHE dataset outperforms the HE dataset for the training accuracy and their training loss.

In addition,

Table 6. Training and testing validation results of the HE and AHE dataset based on sensitivity, specificity, and accuracy.

		Training Set			Testing Set
Dataset	Classifier Method	Percent Sensitivity	Percent Specificity	Percent Accuracy	Percent Sensitivity	Percent Specificity	Percent Accuracy
HE	CNN	94.86%	91.23%	90.25	90.43%	87.16%	88%
	CNN-SVM	96.72%	93.96%	94.16%	93.83%	88.43%	90.16%
AHE	CNN	99%	93.96%	92.96%	92.25%	89%	90%
	CNN-SVM	100%	94.3%	97%	95.55%	90.65%	92.96%

provides the proposed defect detection scheme results on the training and testing of the HE and AHE datasets. It shows the CNN-SVM method is more accurate than the CNN method alone. As shown, the combined approach CNN-SVM outperforms the CNN method with an accuracy of 97% compared to the CNN method’s accuracy of 92.96%, which improves accuracy, and the CNN-SVM is extremely effective in classification. Moreover, the confusion matrix is shown in Figure 22 for the proposed scheme using the AHE dataset.

5. Conclusions

This paper presents an algorithm for thermal imaging technology that extracts the features of thermal images—particularly for solar panel applications—to simplify the representation into an easier domain for analysis. Aside from the histogram of an image, the CDF provides significant insight as one of the parameters when dealing with image processing. The experimental results revealed that the enhanced thermal images provided better histogram and CDF features. Implementing histogram equalization on the dataset provided increased contrast and highlighted the elements of the thermal image. AHE improved the image’s contrast, which could be observed in the corresponding histogram and CDF of the processed thermal images. In addition, the results of this paper provide an improvement on the training accuracy using both the training and testing datasets of the HE and AHE. The result shows that the AHE dataset provides better classification result against the HE dataset when used in the proposed model. It was also validated that the proposed scheme assesses the health condition of the solar panels/cells by the electroluminescence (EL) technique. The proposed work used the convolutional neural network with SVM (CNN-SVM) to classify the enhanced images and showed an accuracy of classification of 97%. Moreover, it shows that the training accuracy and the training loss presented a better result using the proposed CNN-SVM model. The proposed scheme could promote different thermal image applications—for example, non-physical visual recognition and fault detection analysis.

In the future, the different resulting features of these thermal images could contribute to neural networks in classification applications, which could promote the development of different thermal image applications—for example, non-physical visual recognition or the prediction of solar power generation efficiency.