Autonomous Multi-Defect Segmentation in Electroluminescence Images: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor:
Hassan Eesaar
,
Sungjin Joe
,
Mobeen Ur Rehman
,
Yeongmin Jang
,
Kil To Chong

A robust and efficient segmentation framework is essential for accurately detecting and classifying various defects in electroluminescence images of solar photovoltaic (PV) modules. With the increasing global focus on renewable energy resources, solar PV energy systems are gaining significant attention. The inspection of PV modules throughout their manufacturing phase and lifespan requires an automatic and reliable framework to identify multiple micro-defects that are imperceptible to the human eye. 

  • electroluminescence images
  • multi-class semantic segmentation
  • deep learning

1. Introduction

Photovoltaic modules play a crucial role in photovoltaic energy systems, which are part of ongoing efforts to transition toward renewable energy resources. This transition aims to minimize carbon dioxide emissions and mitigate their detrimental effects [1][2]. The International Renewable Energy Agency (IRENA) has taken a firm stance on renewable energy, leading to a global investment of USD 282 billion in the renewable energy sector as of 2019 [3]. This growing momentum toward renewable resources has significantly increased the demand for solar photovoltaic (PV) systems compared to other energy generation systems. However, solar cells may exhibit various defects and shortcomings that can affect the overall energy efficiency of the photovoltaic energy system. Consequently, there is a need to investigate solar cells, starting with their manufacturing phase and conducting inspections throughout their lifespan. Given the emerging trends in energy systems, it is essential to establish a viable and robust assessment mechanism for solar photovoltaic (PV) modules to ensure the anticipated energy harvesting through solar PV energy systems.
Solar PV modules are typically designed with protective measures to withstand different weather conditions and ensure resilience against environmental elements. The front side of the modules is shielded by tempered glass, providing resistance to the stresses and intensities of environmental factors. To safeguard against temperature variations, humidity, and corrosion resulting from water contamination, ethylene vinyl acetate is used as an encapsulation agent [4]. Additionally, a backsheet is incorporated as an additional component to provide mechanical stability, further protection against environmental elements, and insulation for the PV modules [5]. However, despite the presence of these supplementary protective components, multiple defects can occur in the modules over their lifespan. Weather conditions and mechanical damage can contribute to surface defects, while artifacts may also arise during the manufacturing phase [6]. The dynamic temperature and irradiance affect certain parameters of photovoltaic modules, acting as obstacles in estimating these parameters, as the overall throughput of a photovoltaic system hinges on their accurate assessment [7]. The authors propose the L-SHADE and L-SHADED techniques, with the latter method focusing on dimensionality reduction. Following this phase, they employ a linear population size reduction-based success history adaptation differential evolution (L-SHADE) method, consequently determining the unknown parameters. PV modules, such as multi-crystalline KC200GT and mono-crystalline SM55, are utilized under varying temperature and irradiance conditions to identify unknown parameters such as the photo-generated current, series resistance, and diode reverse saturation current. Consequently, inspecting and assessing the condition of PV modules become critical in solar energy systems.
Photovoltaic modules are designed to endure approximately 25 years of continuous exposure to challenging environmental conditions [8]. Manual visual inspection of solar PV modules is a laborious task, and even the scrutiny of a specialist may not be effective, as numerous defects are not visible to the naked eye. In the field of imaging, infrared (IR) cameras can be employed for PV module inspection using infrared imaging. Faulty solar cells that fail to convert solar energy into electrical output emit heat, which can be detected using infrared imaging techniques [9]. However, it is important to note that certain micro-defects may not be captured by infrared imaging, and the limited resolution of infrared cameras poses additional limitations when considering this approach for inspection [10].
Electroluminescence (EL) imaging is considered a preferable alternative to infrared (IR) imaging due to its ability to provide high-quality images capable of capturing micro-defects in solar cells. EL images are obtained by capturing emissions induced at a specific wavelength of 1150 nm using a silicon charge-coupled device (CCD) sensor [11]. In these grayscale EL images, micro-cracks are revealed as dark-gray areas where micro-defects occur [12]. However, visual inspection of EL images is a time-consuming and tedious task, even when performed by an expert. Given the growing trends in renewable energy systems, solar energy plays a major role in this domain. Manual visual assessment is impractical for solar PV energy systems, where it is necessary to inspect thousands of PV modules throughout their lifespan.
The proposed framework aims to achieve efficient EL image segmentation by utilizing a minimal number of model parameters, resulting in a lightweight system. Figure 1 depicts different EL images of solar PV modules, showcasing the presence of multiple co-occurring defects and features.
Figure 1. Three EL images are shown along with their labeled ground-truth masks (a–c). Defects such as cracks, gridlines, and inactive regions and features, such as busbars, ribbons, and padding, are listed in the figure along with their respective color labels.

2. Autonomous Multi-Defect Segmentation in Electroluminescence Images

Electroluminescence (EL) imaging, first experimented with in 2005, has been widely used to capture the degradation patterns of silicon solar cells [13]. In a study by Fuyuki et al. [14], EL imaging was deployed to determine the size of crystalline silicon solar cells and identify cracks and defects, which appeared as darker regions. The study successfully identified inadequate areas in the crystalline silicon solar cells using EL imaging. In another work by Deitsch et al. [15], various defects in mono-crystalline and poly-crystalline PV modules were detected by manually extracting features and employing a Support Vector Machine (SVM) classifier. To further improve defect classification, convolutional neural networks (CNNs) were employed using a dataset of 1968 solar cell images extracted from EL images of PV modules. The CNNs achieved higher classification accuracy compared to the SVM classifier, although the CNNs tended to be more resource-intensive in terms of hardware complexity. Shujaat et al. [16] exploited the practicality of CNNs to identify the promoters responsible for carrying out the transcription of genes.
Similarly, in a comparative study, Karimi et al. [17] evaluated the performance of CNNs against machine learning-based SVM and random forest classifiers. The objective of the study was to categorize solar cell images acquired from EL imaging of PV modules into three defined categories. The framework also incorporated data augmentation techniques to increase the overall dataset size, which consisted of 5400 solar cell images.
In a study by Tsai et al. [18], a Fourier image reconstruction approach was employed to identify defective solar cells using EL images obtained from poly-crystalline PV modules. The method presented in the study provided a reliable means to ascertain the presence of defects in solar cells. Furthermore, in a separate investigation, Anwar et al. [19] combined image segmentation techniques with anisotropic diffuse filtering to detect micro-cracks in solar cells. The study utilized a dataset of 600 EL images and demonstrated the effectiveness of the proposed approach in accurately discerning micro-cracks.
Deep learning has emerged as a potent method for autonomous decision making in several domains, allowing for accurate and fast object and region-of-interest identification [20][21]. In the field of biomedical image segmentation, the U-Net architecture, proposed in [22], has become a widely adopted model due to its effectiveness. It is made up of an encoder–decoder structure that forms a U-shaped network with a contracting and expanding route. Several U-Net design versions have been developed for specialized segmentation tasks in biomedical imaging [23][24]. These include Attention U-Net [25], Dilated Inception U-Net [26], Unet++ [27], SegR-Net [28], RAAGR2-Net [21], and R2U-Net [29]. These variants try to improve the U-Net model’s performance in certain scenarios. DeepLabv3+ is another well-known semantic segmentation framework, which expands the DeepLabv3 paradigm [30] by combining encoder–decoder blocks with Xception and ResNet-101 as the network backbones [31]. This architecture was created specifically for semantic segmentation tasks and has performed well. In biomedical imaging, dual-encoder- and dual-decoder-based DL frameworks are employed for aiding in the diagnosis of colorectal cancer through polyp segmentation from colonoscopy images [32]. In another work, Wooram et al. [33] adopted a neural network based on convolutional operations, along with atrous spatial pyramid pooling and separable convolutions integrated with a decoder module, to carry out the real-time segmentation of external cracks in structures.
The literature includes a variety of deep learning approaches aimed at classifying cells and modules into a wide range of flaws while accounting for the varied severity of these problems in solar PV modules [34][35]. In one work, Rahman et al. [36] adapted the U-Net architecture to identify flaws in poly-crystalline solar cells using EL pictures. The study proved the modified U-Net’s accuracy in segmenting and finding flaws in solar cells. Deqiang et al. [37] proposed a U-Net-based framework for single image super-resolution (SISR), with the sole purpose of image reconstruction from bristly textures to finer details. The approach is termed anti-illumination, as it effectively subdues the noise in images while captivating the illuminance details.
Another paper [38] by Chen et al. reported the segmentation of fractures in EL pictures of multi-crystalline solar cells. The study presented a unique approach for identifying and isolating fractures in photographs, which helped to characterize the faults. In order to extract global features, Ruixuan et al. [39] introduced a transformer-based architecture, LF-DET, for light-field spatial super-resolution. The model comprises two subsets. The first part introduces convolutional layers before self-attention modules, consequently obtaining global features. Further, in the second part, feature representations from macro areas are obtained at various levels using angular modeling across multiple scales. Furthermore, Pratt et al. [40] used EL pictures of PV modules to detect flaws in mono-crystalline and multi-crystalline silicon solar cells through a U-Net-based image semantic segmentation framework. The suggested framework yielded encouraging results in detecting and segmenting faults in solar cells. For crack segmentation, Young et al. [41] proposed a convolutional neural network-based DL model, comparing it to image processing edge detection approaches such as Canny and Sobel detection. Another work by Young et al. [42] reported the adaption of a faster region-based convolutional neural network (Faster R-CNN) for automatically detecting five different types of damage in the visual probing of structures.

This entry is adapted from the peer-reviewed paper 10.3390/en16237726

References

  1. Tu, Q.; Mo, J.; Betz, R.; Cui, L.; Fan, Y.; Liu, Y. Achieving grid parity of solar PV power in China-The role of Tradable Green Certificate. Energy Policy 2020, 144, 111681.
  2. Adams, S.; Nsiah, C. Reducing carbon dioxide emissions; Does renewable energy matter? Sci. Total. Environ. 2019, 693, 133288.
  3. Yang, Z.; Zhang, M.; Liu, L.; Zhou, D. Can renewable energy investment reduce carbon dioxide emissions? Evidence from scale and structure. Energy Econ. 2022, 112, 106181.
  4. Peike, C.; Hädrich, I.; Weiß, K.A.; Dürr, I.; Ise, F. Overview of PV module encapsulation materials. Photovoltaics Int. 2013, 19, 85–92.
  5. Makrides, G.; Theristis, M.; Bratcher, J.; Pratt, J.; Georghiou, G.E. Five-year performance and reliability analysis of monocrystalline photovoltaic modules with different backsheet materials. Sol. Energy 2018, 171, 491–499.
  6. Haque, A.; Bharath, K.V.S.; Khan, M.A.; Khan, I.; Jaffery, Z.A. Fault diagnosis of photovoltaic modules. Energy Sci. Eng. 2019, 7, 622–644.
  7. Gu, Q.; Li, S.; Gong, W.; Ning, B.; Hu, C.; Liao, Z. L-SHADE with parameter decomposition for photovoltaic modules parameter identification under different temperature and irradiance. Appl. Soft Comput. 2023, 143, 110386.
  8. Makrides, G.; Zinsser, B.; Schubert, M.; Georghiou, G.E. Performance loss rate of twelve photovoltaic technologies under field conditions using statistical techniques. Sol. Energy 2014, 103, 28–42.
  9. Buerhop, C.; Bommes, L.; Schlipf, J.; Pickel, T.; Fladung, A.; Peters, M. Infrared imaging of photovoltaic modules A review of the state of the art and future challenges facing gigawatt photovoltaic power stations. Prog. Energy 2022, 4, 042010.
  10. Rahaman, S.A.; Urmee, T.; Parlevliet, D.A. PV system defects identification using Remotely Piloted Aircraft (RPA) based infrared (IR) imaging: A review. Sol. Energy 2020, 206, 579–595.
  11. Fuyuki, T.; Tani, A. Photographic diagnosis of crystalline silicon solar cells by electroluminescence. In Experimental and Applied Mechanics, Volume 6: Proceedings of the 2010 Annual Conference on Experimental and Applied Mechanics; Springer: Berlin/Heidelberg, Germany, 2011; pp. 159–162.
  12. Breitenstein, O.; Bauer, J.; Bothe, K.; Hinken, D.; Müller, J.; Kwapil, W.; Schubert, M.C.; Warta, W. Can luminescence imaging replace lock-in thermography on solar cells? IEEE J. Photovoltaics 2011, 1, 159–167.
  13. Fuyuki, T.; Kondo, H.; Yamazaki, T.; Takahashi, Y.; Uraoka, Y. Photographic surveying of minority carrier diffusion length in polycrystalline silicon solar cells by electroluminescence. Appl. Phys. Lett. 2005, 86, 262108.
  14. Fuyuki, T.; Kitiyanan, A. Photographic diagnosis of crystalline silicon solar cells utilizing electroluminescence. Appl. Phys. A 2009, 96, 189–196.
  15. Deitsch, S.; Christlein, V.; Berger, S.; Buerhop-Lutz, C.; Maier, A.; Gallwitz, F.; Riess, C. Automatic classification of defective photovoltaic module cells in electroluminescence images. Sol. Energy 2019, 185, 455–468.
  16. Shujaat, M.; Wahab, A.; Tayara, H.; Chong, K.T. pcPromoter-CNN: A CNN-based prediction and classification of promoters. Genes 2020, 11, 1529.
  17. Karimi, A.M.; Fada, J.S.; Hossain, M.A.; Yang, S.; Peshek, T.J.; Braid, J.L.; French, R.H. Automated pipeline for photovoltaic module electroluminescence image processing and degradation feature classification. IEEE J. Photovoltaics 2019, 9, 1324–1335.
  18. Tsai, D.M.; Wu, S.C.; Li, W.C. Defect detection of solar cells in electroluminescence images using Fourier image reconstruction. Sol. Energy Mater. Sol. Cells 2012, 99, 250–262.
  19. Anwar, S.A.; Abdullah, M.Z. Micro-crack detection of multicrystalline solar cells featuring an improved anisotropic diffusion filter and image segmentation technique. Eurasip J. Image Video Process. 2014, 2014, 15.
  20. Rehman, M.U.; Akhtar, S.; Zakwan, M.; Mahmood, M.H. Novel architecture with selected feature vector for effective classification of mitotic and non-mitotic cells in breast cancer histology images. Biomed. Signal Process. Control 2022, 71, 103212.
  21. Rehman, M.U.; Ryu, J.; Nizami, I.F.; Chong, K.T. RAAGR2-Net: A brain tumor segmentation network using parallel processing of multiple spatial frames. Comput. Biol. Med. 2023, 152, 106426.
  22. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Proceedings, Part III 18. Springer: Berlin/Heidelberg, Germany, 2015; pp. 234–241.
  23. Rehman, M.U.; Cho, S.; Kim, J.; Chong, K.T. Brainseg-net: Brain tumor mr image segmentation via enhanced encoder–decoder network. Diagnostics 2021, 11, 169.
  24. Rehman, M.U.; Cho, S.; Kim, J.H.; Chong, K.T. Bu-net: Brain tumor segmentation using modified u-net architecture. Electronics 2020, 9, 2203.
  25. Oktay, O.; Schlemper, J.; Folgoc, L.L.; Lee, M.; Heinrich, M.; Misawa, K.; Mori, K.; McDonagh, S.; Hammerla, N.Y.; Kainz, B.; et al. Attention u-net: Learning where to look for the pancreas. arXiv 2018, arXiv:1804.03999.
  26. Cahall, D.E.; Rasool, G.; Bouaynaya, N.C.; Fathallah-Shaykh, H.M. Dilated inception U-net (DIU-net) for brain tumor segmentation. arXiv 2021, arXiv:2108.06772.
  27. Zhou, Z.; Rahman Siddiquee, M.M.; Tajbakhsh, N.; Liang, J. Unet++: A nested u-net architecture for medical image segmentation. In Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 20 September 2018; Proceedings 4. Springer: Berlin/Heidelberg, Germany, 2018; pp. 3–11.
  28. Ryu, J.; Rehman, M.U.; Nizami, I.F.; Chong, K.T. SegR-Net: A deep learning framework with multi-scale feature fusion for robust retinal vessel segmentation. Comput. Biol. Med. 2023, 163, 107132.
  29. Alom, M.Z.; Hasan, M.; Yakopcic, C.; Taha, T.M.; Asari, V.K. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv 2018, arXiv:1802.06955.
  30. Chen, L.C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587.
  31. Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818.
  32. Lewis, J.; Cha, Y.J.; Kim, J. Dual encoder–decoder-based deep polyp segmentation network for colonoscopy images. Sci. Rep. 2023, 13, 1183.
  33. Choi, W.; Cha, Y.J. SDDNet: Real-time crack segmentation. IEEE Trans. Ind. Electron. 2019, 67, 8016–8025.
  34. Demirci, M.Y.; Beşli, N.; Gümüşçü, A. Efficient deep feature extraction and classification for identifying defective photovoltaic module cells in Electroluminescence images. Expert Syst. Appl. 2021, 175, 114810.
  35. Tang, W.; Yang, Q.; Hu, X.; Yan, W. Convolution neural network based polycrystalline silicon photovoltaic cell linear defect diagnosis using electroluminescence images. Expert Syst. Appl. 2022, 202, 117087.
  36. Rahman, M.R.U.; Chen, H. Defects inspection in polycrystalline solar cells electroluminescence images using deep learning. IEEE Access 2020, 8, 40547–40558.
  37. Cheng, D.; Chen, L.; Lv, C.; Guo, L.; Kou, Q. Light-Guided and Cross-Fusion U-Net for Anti-Illumination Image Super-Resolution. IEEE Trans. Circuits Syst. Video Technol. 2022, 32, 8436–8449.
  38. Chen, H.; Zhao, H.; Han, D.; Liu, K. Accurate and robust crack detection using steerable evidence filtering in electroluminescence images of solar cells. Opt. Lasers Eng. 2019, 118, 22–33.
  39. Cong, R.; Sheng, H.; Yang, D.; Cui, Z.; Chen, R. Exploiting Spatial and Angular Correlations with Deep Efficient Transformers for Light Field Image Super-Resolution. IEEE Trans. Multimed. 2023, 1–14.
  40. Pratt, L.; Govender, D.; Klein, R. Defect detection and quantification in electroluminescence images of solar PV modules using U-net semantic segmentation. Renew. Energy 2021, 178, 1211–1222.
  41. Cha, Y.J.; Choi, W.; Büyüköztürk, O. Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 2017, 32, 361–378.
  42. Cha, Y.J.; Choi, W.; Suh, G.; Mahmoudkhani, S.; Büyüköztürk, O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput.-Aided Civ. Infrastruct. Eng. 2018, 33, 731–747.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
Video Production Service
Feedback