AI and XAI for Visual Quality Assurance

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Rudolf Hoffmann	--	2738	2024-02-01 09:08:36	\|
2	Reference format revised.	Lindsay Dong	+ 2 word(s)	2740	2024-02-02 07:39:22	\|

This entry is adapted from the peer-reviewed paper 10.3390/electronics12224572

Quality assurance (QA) plays a crucial role in manufacturing to ensure that products meet their specifications. However, manual QA processes are costly and time-consuming, thereby making artificial intelligence (AI) an attractive solution for automation and expert support. In particular, convolutional neural networks (CNNs) have gained a lot of interest in visual inspection. Next to AI methods, the explainable artificial intelligence (XAI) systems, which achieve transparency and interpretability by providing insights into the decision-making process of the AI, are interesting methods for achieveing quality inspections in manufacturing processes.

XAI AI machine learning deep learning image processing interpretability

1. Introduction

In the present globalized economic era, the competition in the business world requires that management teams of various organizations and businesses must constantly drive along the quality route to outpace each other ^[1]. Monitoring and assessing the manufacturing process is an important factor for manufacturers to detect potential failures that may lead to the degradation of the machinery of manufacturing of downgraded products and thus assure a constant product quality ^[2]. To ensure that a product is defect-free before leaving the factory, some type of quality assurance (QA) practices is necessary. Quality control (QC) is a subset of QA and it is mostly used to assure quality in products. Artificial intelligence (AI) can help manufacturers perform QA more accurately and cost-effectively by automating QC ^[3]. Moreover, as the manufacturing industry seeks innovative QA approaches, the emergence of paradigms such as Zero Defect Manufacturing (ZDM) is gaining traction. ZDM represents a transformative approach aimed at eliminating defects throughout the production process, thereby aligning with the broader industry’s pursuit of superior quality and efficiency ^[4]. AI applications have made progress in solving the automatic recognition of patterns in data by using machine learning (ML) and deep learning (DL) methods. These AI-based systems not only streamline QC processes, but also have the potential to seamlessly integrate with evolving QA practices aimed at achieving the core strategies of ZDM. Such integration enables manufacturers to perform QA more effectively by harnessing AI’s capabilities for precise defect detection, process optimization, predictive maintenance, and root cause analysis, all while reducing costs and enhancing overall product quality. Data-driven decision making may influence meaningful productivity gains in the industry sectors; however, for the stable deployment of AI-based systems and their acceptance by experts and regulators, decisions and results must be comprehensible or interpretable and transparent; in other words, they must be “Explainable” ^[5].

In recent years, explainable artificial intelligence (XAI) has emerged as a promising solution to these challenges, thus providing transparency and interpretability to AI systems. The goal of XAI is to help researchers, developers, domain experts, and users better understand the inner operation of ML models while preserving their high performance and accuracy. XAI methods seek to understand what the AI-based system discovered during training and how decisions are made for specific or new occurrences during the prediction process ^[6]^[7]. This can help to understand why predictions went wrong or why ML achieves a specific result and how to leverage the result further.

2. AI and XAI for Visual Quality Assurance in Manufacturing

2.1. Artificial Intelligence Facilitates Visual Quality Assurance

AI is a new type of technological science that investigates and develops theories, methods, technologies, and application systems to simulate, improve, and upgrade human intelligence. It has been created to enable machines to reason like human beings and to endow them with intelligence. AI systems aim to possess characteristics such as problem-solving, pattern recognition, and decision-making capabilities. By leveraging techniques such as ML and computer vision, AI seeks to create intelligent machines that can process and interpret vast amounts of data, recognize patterns, adapt to changing circumstances, and perform complex tasks ^[8]^[9]. ML is already applied to solve real problems in different domains, such as medicine ^[10], handwriting recognition ^[11], and manufacturing ^[12]. The following AI approaches are typically used for visual QA (VQA):

Classification (Class): In this approach, the algorithm learns to predict discrete class labels for input data and thus can facilitate VQA by categorizing data, products, or processes ^[8]. It can automate VQC and ensure that only defect-free products are delivered. Classification can help in identifying areas for improvement by categorizing defects, reducing waste, and saving energy by identifying defects in early process stages ^[13]. By inspecting images of machine spare parts, wear can be identified and the spare parts can be replaced to prevent the quality losses of the manufactured products ^[14].
Estimation (Est): Estimation refers to the process of predicting or approximating an unknown or future value based on available data. It involves making an inference or calculation to estimate a parameter or outcome ^[8]. By analyzing historical data, the most suitable processing parameters ^[15], the remaining error-free time ^[16], or the defect length can be predicted ^[17].
Object Detection (OD): Object detection is a computer vision task that involves identifying and localizing objects within images or videos. It combines object classification (assigning labels to objects) and object localization (drawing bounding boxes around objects). Object detection algorithms can accurately detect and locate multiple objects or flaws in products ^[18]. Thus, objection detection is a more comprehensive approach than classification, because occuring or predicting flaws can be additionally localized.
Segmentation (Seg): Segmentation is a computer vision task that involves dividing images or videos into distinct regions based on visual characteristics. Similar to object detection, it can be used to localize defects or anomalies within products or processes, thereby ensuring accurate inspections and prompt corrective actions ^[19].
Anomaly Detection (AD): In anomaly detection, the algorithm learns the normal patterns in the data and identifies any data points that deviate significantly from those patterns, thus often indicating anomalies or outliers ^[20]. This method can be used, for example, to monitor image or image-like data in manufacturing and take corrective actions promptly if potential issues or anomalies are identified ^[21]. In VQA, it can be used to detect rare or abnormal occurences in images, thus helping organizations to make data-driven decisions to improve processes.

2.2. Explainable Artificial Intelligence Facilitates Quality Assurance

In conventional AI systems, the learning process cannot be interpreted by the end-users, and it looks like an opaque black box. XAI, on the other hand, specifically emphasizes the need for transparency and interpretability in AI systems. By explaining the relationships between input variables and quality outcomes, it helps identify the underlying causes of quality losses. It aims to address the black box nature of many AI models, where their decision-making processes are often difficult to understand or explain. XAI focuses on developing techniques and methods that enable humans to understand and interpret the results and inner workings of AI models. The ability to understand the mechanism of decision making of a model is an important factor for three main reasons. First, it makes it possible to further refine and improve its analysis. Second, it becomes easier to explain to nondata scientists the way in which the model uses the data to make decisions. Third, explainability can help data scientists to avoid the negative or unforeseen consequences of their models. Figure 1 shows a simple taxonomy.

Figure 1. XAI taxonomy for the methods.

Intrinsic Explanation: The intrinsic explanation techniques enable the extraction of decision rules directly from the model’s architecture. ML algorithms like linear regression (LR), logistic regression (LogR), decision trees (DTs), and rule-based models are frequently employed to create intrinsic models ^[22].
Post Hoc Explanation: Post hoc explanation techniques are designed to uncover the relationships between feature values and predictions. In contrast, models such as deep neural networks are often less interpretable, as they do not readily yield explicit decision rules from their structural components.
Model-Specific Explanation (Mod-Sp): Model-specific explanation techniques in XAI are tailored to particular contexts and conditions, thus leveraging the unique characteristics of the underlying algorithm or the specific architecture of an AI model. Reverse engineering approaches are applied to probe the internals of the algorithms. For example, class activation mapping (CAM) or gradient-weighted CAM (GradCAM) methods offer visual explanations specifically designed for CNN models. These CAM-based methods generate localization maps from convolutional layers, thereby revealing the crucial image regions that contribute to predicting a particular concept ^[22].
Model-Agnostic Explanation (Mod-Ag): Model-agnostic explanation techniques focus on the relations between feature values and prediction results. These methods do not depend on or make assumptions about the specific ML model or algorithm being explained. These methods are designed to provide interpretability and explanations for a wide range of ML models, thus making them versatile and applicable across various domains ^[22].

2.3. Explainability of Visual Quality Assurance Processes

The application of XAI approaches to VQA processes can enhance transparency, interpretability, and trust in AI systems applied to VQA.

Transparency in Visual Quality Assurance Practices: XAI provides transparency by offering clear insights into the decision-making processes of AI models used for VQA. Transparency is essential for assessing the quality of a model’s decision ^[23]. For example, ref. ^[24] used the GradCAM method to visualize the regions on which a CNN model is most focusing its attention for decision making to provide a human-readable explanation of the CNN model’s decision-making process.
Interpretability of Visual Quality Assurance Practices: Interpretability refers to the ability to understand the underlying workings of an AI model. It involves comprehending how AI models make their decisions. Interpretability can be achieved through techniques that explain the internals of an AI model in a manner that humans can comprehend. These techniques are known as intrinsic methods ^[23]. In ^[25], intrinsic interpretable tree-based models were used to assure quality. Interpretability helps users comprehend how various factors impact product quality and enables the identification of the most influential parameters in a manufacturing process, thus allowing for better optimization and control.
Trust in Visual Quality Assurance Decisions: When the decision-making process in a model is thoroughly understood, the model becomes transparent. Transparency promotes trust in the model ^[23]. This collaboration between humans and AI models enhances the efficiency and accuracy of VQA processes, thereby leading to more reliable results.

2.4. Role of AI and XAI in VQA

Visual Quality Control

The comparitive study in ^[3] has already explained how AI-based visual inspection can empower VQC activities. With AI, automated visual product inspections can be realized more accurately and cost-effectively to monitor products or processes for faults, contaminations, and other anomalies. AI-based visual inspection achieves the following advantages:

It is not biased by the operator’s viewpoint;
It can be adjusted to products changes, and no programming is necessary;
It is not quickly weary;
It is fast;
It can see a wide spectrum of colors;
It can operate in potentially dangerous settings;
Its operators have fewer cognitive burdens.

Most of the literature found in this survey dealt with VQC in the manufacturing process to detect defects. Most approaches used DL methods, such as pretrained CNNs, that automatically extract features from images, and only the classifier has to be replaced to classify the quality of the products of these images. For example, ^[26] used a pretrained VGG16 model and a pretrained XCeption model to identify defects. Afterwards, an XAI method called GradCAM was applied to this model to visualize the area that most contributed to the prediction. The study ^[27] built a VGG-like CNN model with 17 layers that achieved higher testing accuracy than other models, such as SVM. In order to gain insights into the interpretability of this CNN model, the authors employed CAM. This technique facilitated the generation of a heatmap, thus highlighting the regions that significantly influence the model’s decision-making process. While GradCAM and CAM are model-specific XAI approaches, the study of ^[28] used a model-agnostic XAI approach to provide interpretability to a CNN model. It used the SHAP method to understand the contribution of individual pixels for the outcome. These XAI methods have several advantages. On the one hand, the data scientist who develops the AI model can evaluate the model’s behavior more easily and create a more robust and reliable model. On the other hand, the model becomes more transparent. It provides explanations, justifications, and insights into the factors and features that influenced the outcome. This helps QA professionals and auditors to interpret how the AI model arrived at its decisions and identify potential errors or biases in the AI models. Furthermore, they can trust the model, since XAI addresses the black box characteristic of the AI model. The model-specific methods GradCAM and CAM highlight the regions that most contribute to the prediction. The model-agnostic method SHAP provides information about the contribution of each individual pixel for the outcome. Thus, XAI flaws can be revealed, and the stakeholders can verify the model’s prediction when quality issues are detected.

Process Optimization

AI can help optimize the process parameters to improve the quality of manufactured products ^[15]. When there are multiple process parameters involved, the complexity of the outcome prediction increases, because more relationships and interactions among the variables have to be considered. Here, ML methods can be used to solve the complex task. A possible solution is to use process parameters as input parameters and predict the corresponding output or the desired quality. Using XAI helps to identify features that are most important for the quality. Varying these features or parameters has the greatest impact on quality.

The study in ^[29] focused on monitoring multilayer optical tomography images to predict local porosity in additive manufacturing. Random forest (RF) was utilized for porosity prediction, providing interpretability, and identifying the most important features (layers) contributing to the prediction. The proposed model not only predicted porosity, but also indicated the optimal processing window for achieving near-zero porosity.

Overall, these studies highlight the application of ML and DL techniques for monitoring, optimizing, and regulating manufacturing processes parameters. The incorporation of interpretability, multimodality, and adaptive control mechanisms demonstrates the potential for improving quality. Indeed, optimizing process parameters solely based on visual data may not be sufficient in certain scenarios. While visual data can provide valuable insights into the visual aspects of a process, there are other factors and variables that may need to be considered for effective regulation. For the optimization of process parameters, it is essential to take into account multiple sources of data, including but not limited to visual data. Depending on the specific manufacturing process, additional data sources such as sensor data (e.g., temperature, pressure), historical process data, or real-time feedback from the production line may be necessary. By integrating multiple data sources, a more holistic understanding of the process can be achieved, thereby allowing for a more accurate and robust regulation of process parameters. This comprehensive approach enables the consideration of factors beyond visual defects, such as material properties, environmental conditions, and overall process stability.

Predictive Maintenance

In PM, information is gathered in real time to control the status of devices or machines. The aim is to discover patterns that can assist in predicting and eventually anticipating malfunctions ^[5]. Machineries have to be maintained in time to ensure consistent quality of the manufactured products. However, maintenance is required with additional cost. Thus, the machine should be optimally maintained just before the entrance of the malfunction or before the wear of the machine causes insufficient quality of the product. There are several AI approaches that help in achieving PM. XAI can provide interpretable insights into the prediction process of anomalies that lead to quality deviations. This helps manufacturers detect potential problems at an early stage, thereby enabling timely intervention and corrective actions. Ref. ^[30] used regression CNN models on time series data for PM. Then, they applied different XAI methods to explore their performance. In terms of the methods LIME, SHAP, LRP, GradCAM, and Image-Specific Class Saliency (ISCSal), GradCAM achieved best performance.

Root Cause Analysis

Applying VQA into the organization enables the continual improvement of the quality by meeting the VQA practices. A further approach to improve quality is the identification of root causes when a malfunction or anomaly has been detected. The identification of root causes aims to understand the underlying reasons behind these malfunctions or anomalies. By uncovering the root cause, organizations can address and eliminate the problem, thereby leading to improved product quality and a reduction in future malfunctions identified during VQC. To identify the root causes of anomalies, XAI methods can be employed. By understanding which features or factors contribute most to the detection of anomalies, organizations can gain a deeper understanding of the root causes and take appropriate corrective actions. By incorporating XAI methods into the VQC, organizations can not only identify and resolve specific issues, but also enhance overall product quality and consistency. This proactive approach helps to minimize the occurrence of future malfunctions, thus resulting in improved customer satisfaction, increased operational efficiency, and reduced costs associated with quality issues.

References

Ibidapo, T.A. From Industry 4.0 to Quality 4.0—An Innovative TQM Guide for Sustainable Digital Age Businesses; Springer: Cham, Switzerland, 2022.
Jakubowski, J.; Stanisz, P.; Bobek, S.; Nalepa, G.J. Explainable anomaly detection for Hot-rolling industrial process. In Proceedings of the 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), Porto, Portugal, 6–9 October 2021; pp. 1–10.
Arora, A.; Gupta, R. A Comparative Study on Application of Artificial Intelligence for Quality Assurance in Manufacturing. In Proceedings of the 2022 4th International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 21–23 September 2022; pp. 1200–1206.
Psarommatis, F.; Sousa, J.; Mendonça, J.P.; Kiritsis, D. Zero-defect manufacturing the approach for higher manufacturing sustainability in the era of industry 4.0: A position paper. Int. J. Prod. Res. 2022, 60, 73–91.
Ahmed, I.; Jeon, G.; Piccialli, F. From Artificial Intelligence to Explainable Artificial Intelligence in Industry 4.0: A Survey on What, How, and Where. IEEE Trans. Ind. Inform. 2022, 18, 5031–5042.
Tabassum, S.; Parvin, N.; Hossain, N.; Tasnim, A.; Rahman, R.; Hossain, M.I. IoT Network Attack Detection Using XAI and Reliability Analysis. In Proceedings of the 2022 25th International Conference on Computer and Information Technology (ICCIT), Cox’s Bazar, Bangladesh, 17–19 December 2022; pp. 176–181.
Machlev, R.; Heistrene, L.; Perl, M.; Levy, K.; Belikov, J.; Mannor, S.; Levron, Y. Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities. Energy AI 2022, 9, 100169.
Huawei Technologies Co., Ltd. Artificial Intelligence Technology; Springer: Singapore, 2023.
Szeliski, R. Computer Vision—Algorithms and Applications; Springer: Berlin/Heidelberg, Germany, 2022.
Stojanov, D.; Lazarova, E.; Veljkova, E.; Rubartelli, P.; Giacomini, M. Predicting the outcome of heart failure against chronic-ischemic heart disease in elderly population – Machine learning approach based on logistic regression, case to Villa Scassi hospital Genoa, Italy. J. King Saud Univ.-Sci. 2023, 35, 102573.
Pesch, H.; Hamdani, M.; Forster, J.; Ney, H. Analysis of Preprocessing Techniques for Latin Handwriting Recognition. In Proceedings of the 2012 International Conference on Frontiers in Handwriting Recognition, Bari, Italy, 18–20 September 2012; pp. 280–284.
Chen, T.; Sampath, V.; May, M.C.; Shan, S.; Jorg, O.J.; Aguilar Martín, J.J.; Stamer, F.; Fantoni, G.; Tosello, G.; Calaon, M. Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to ‘Four-Know’. Appl. Sci. 2023, 13, 1903.
Kumar, R.; Patil, O.; Nath S, K.; Sangwan, K.S.; Kumar, R. A Machine Vision-based Cyber-Physical Production System for Energy Efficiency and Enhanced Teaching-Learning Using a Learning Factory. Procedia CIRP 2021, 98, 424–429.
Pheng, M.S.K.; David, L.G. Artificial Intelligence in Back-End Semiconductor Manufacturing: A Case Study. In Proceedings of the 2022 IEEE International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), Ballari, India, 23–24 April 2022; pp. 1–4.
Rehman, R.U.; Zaman, U.K.U.; Aziz, S.; Jabbar, H.; Shujah, A.; Khaleequzzaman, S.; Hamza, A.; Qamar, U.; Jung, D.W. Process Parameter Optimization of Additively Manufactured Parts Using Intelligent Manufacturing. Sustainability 2022, 14, 15475.
Bauer, M.; Augenstein, C.; Schäfer, M.; Theile, O. Artificial Intelligence in Laser Powder Bed Fusion Procedures—Neural Networks for Live-Detection and Forecasting of Printing Failures. Procedia CIRP 2022, 107, 1367–1372.
Rodríguez-Martín, M.; Fueyo, J.; Pisonero, J.; López-Rebollo, J.; Gonzalez-Aguilera, D.; García-Martín, R.; Madruga, F. Step heating thermography supported by machine learning and simulation for internal defect size measurement in additive manufacturing. Measurement 2022, 205, 112140.
Prezas, L.; Michalos, G.; Arkouli, Z.; Katsikarelis, A.; Makris, S. AI-enhanced vision system for dispensing process monitoring and quality control in manufacturing of large parts. Procedia CIRP 2022, 107, 1275–1280.
Yao, K.; Ortiz, A.; Bonnin-Pascual, F. A Weakly-Supervised Semantic Segmentation Approach Based on the Centroid Loss: Application to Quality Control and Inspection. IEEE Access 2021, 9, 69010–69026.
Susto, G.A.; Terzi, M.; Beghi, A. Anomaly Detection Approaches for Semiconductor Manufacturing. Procedia Manuf. 2017, 11, 2018–2024.
Maggipinto, M.; Beghi, A.; Susto, G.A. A Deep Convolutional Autoencoder-Based Approach for Anomaly Detection with Industrial, Non-Images, 2-Dimensional Data: A Semiconductor Manufacturing Case Study. IEEE Trans. Autom. Sci. Eng. 2022, 19, 1477–1490.
Huang, J.; Wang, Z.; Li, D.; Liu, Y. The Analysis and Development of an XAI Process on Feature Contribution Explanation. In Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan, 17–20 December 2022; pp. 5039–5048.
Ali, S.; Abuhmed, T.; El-Sappagh, S.; Muhammad, K.; Alonso-Moral, J.M.; Confalonieri, R.; Guidotti, R.; Del Ser, J.; Díaz-Rodríguez, N.; Herrera, F. Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence. Inf. Fusion 2023, 99, 101805.
Klamert, V.; Schmid-Kietreiber, M.; Bublin, M. A deep learning approach for real time process monitoring and curling defect detection in Selective Laser Sintering by infrared thermography and convolutional neural networks. Procedia CIRP 2022, 111, 317–320.
Huang, D.J.; Li, H. A machine learning guided investigation of quality repeatability in metal laser powder bed fusion additive manufacturing. Mater. Des. 2021, 203, 109606.
Westphal, E.; Seitz, H. A machine learning method for defect detection and visualization in selective laser sintering based on convolutional neural networks. Addit. Manuf. 2021, 41, 101965.
Lee, S.Y.; Tama, B.A.; Moon, S.J.; Lee, S. Steel Surface Defect Diagnostics Using Deep Convolutional Neural Network and Class Activation Map. Appl. Sci. 2019, 9, 5449.
Sheu, R.K.; Chen, L.C.; Pardeshi, M.S.; Pai, K.C.; Chen, C.Y. AI Landing for Sheet Metal-Based Drawer Box Defect Detection Using Deep Learning (ALDB-DL). Processes 2021, 9, 768.
Feng, S.; Chen, Z.; Bircher, B.; Ji, Z.; Nyborg, L.; Bigot, S. Predicting laser powder bed fusion defects through in-process monitoring data and machine learning. Mater. Des. 2022, 222, 111115.
Solís-Martín, D.; Galán-Páez, J.; Borrego-Díaz, J. On the Soundness of XAI in Prognostics and Health Management (PHM). Information 2023, 14, 256.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Computer Science, Artificial Intelligence

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Rudolf Hoffmann

Christoph Reich

View Times: 223

Update Date: 02 Feb 2024

Table of Contents

Video Upload Options

Confirm