Anomaly Detection System for Automatic Defective Products’ Inspection: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Camila Xu and Version 1 by Yu-Hsin Hung.

An automated optical inspection (AOI) system is an efficient tool for product inspection, providing a convenient interface for users to view their products of interest. Specifically, in the screw manufacturing industry, the conventional methods are the human visual inspection of the product and for the inspector to view the product image displayed on the dashboard of the AOI system.

  • deep learning
  • image processing

1. Introduction

A screw product comprises a cap head and a thread. Since screws are critical equipment components, a faulty one can cause major problems, such as the equipment failing to operate normally. Hence, screw inspection is critical for its manufacturer. The screw inspection process was divided into four stages. The first stage was to inspect the surface of the cap head; the second was to inspect the screw’s roundness; the third was to inspect the screw thread; and the fourth was to inspect the screw eccentricity (Figure 1). An automated optical inspection (AOI) system is a visual tool that uses a camera to automatically photograph a product during its inspection. In this case, the inspector also watched the photographed image on an AOI dashboard to further detect any defective surface on the cap head. Notwithstanding, the inspector’s subjective assessment still determines whether or not the product is defective, and sometimes the inspector may incorrectly conduct a defective inspection.Industry 4.0 aims to enable digitalization, intelligentization, interconnectivity, and automation using cloud technologies, cyber-physical systems (CPS), artificial intelligence (AI), and machine learning (ML). In CPS, internet of things (IoT)-related tools (e.g., sensors, internet-connected devices, and software applications) are used to smart wire objects for industrial applications. Consequently, massive industrial data can be obtained within the CPS environment during internet-connected manufacturing. Previously, Kamat and Sugandhi [1] investigated the correlation between anomaly detection and predictive maintenance in Industry 4.0. They observed that since a human visual inspection hardly identifies anomaly events and rare observations for anomaly detection, data collected from CPS could be analyzed to automatically find defected patterns. Alternatively, according to IBM, automation includes information technology, business processes, and other types of automation [2]. On this basis, another study reported that data-driven anomaly detection could improve inspection efficiency [3], and cloud technologies have recently been widely applied to automation applications. Moreover, automating all or part of the manual tasks using specialized software and methodologies can enable auto-notification while detecting an anomaly. Hence, cloud technologies, such as the message queuing telemetry transport (MQTT) protocol, application programming interface (API), and web programming, have recently been applied widely to automatically function in human and machine communication. Industry 4.0 aims to enable digitalization, intelligentization, interconnectivity, and automation using cloud technologies, cyber-physical systems (CPS), artificial intelligence (AI), and machine learning (ML). In CPS, internet of things (IoT)-related tools (e.g., sensors, internet-connected devices, and software applications) are used to smart wire objects for industrial applications. Consequently, massive industrial data can be obtained within the CPS environment during internet-connected manufacturing. Previously, Kamat and Sugandhi [1] investigated the correlation between anomaly detection and predictive maintenance in Industry 4.0. They observed that since a human visual inspection hardly identifies anomaly events and rare observations for anomaly detection, data collected from CPS could be analyzed to automatically find defected patterns. Alternatively, according to IBM, automation includes information technology, business processes, and other types of automation [2]. On this basis, another study reported that data-driven anomaly detection could improve inspection efficiency [3], and cloud technologies have recently been widely applied to automation applications. Moreover, automating all or part of the manual tasks using specialized software and methodologies can enable auto-notification while detecting an anomaly. Hence, cloud technologies, such as the message queuing telemetry transport (MQTT) protocol, application programming interface (API), and web programming, have recently been applied widely to automatically function in human and machine communication.
Although improving product quality in the screw manufacturing process was previously demonstrated using the above-mentioned technologies, this study used the visual geometry group network (VGG-16), Inception V3, and Xception algorithms for real industrial image datasets. Then, the MQTT protocol, API, and web programming methods were employed in the proposed system to develop cloud dashboards, auto-notifications, and data communication systems.

2. Anomaly Detection System for Automatic Defective Products’ Inspection

Related literature demonstrates that machine learning, data mining, mathematical methods, and deep learning approaches provide users with a convenient way to analyze data from manufacturing sites for anomaly detection (Table 1). Table 1 lists the current approaches for anomaly detection. Some anomaly detection-related research used machine-learning approaches. For example, Winters et al. [4] investigated control charts and autoregressive models to detect anomalies for predictive maintenance; Erdmann [5] used an unsupervised approach to recognize anomalous sensor data for predictive maintenance; Minarini [6] proposed a framework for detecting anomalies in log-based predictive maintenance; Alaoui-Belghiti et al. [7] proposed unsupervised online methods using optimal transport to recognize the anomalies for predictive maintenance; Farbiz et al. [8] proposed cognitive analytics with unsupervised learning to predict the machine status for equipment health maintenance; Carrasco et al. [9] proposed a framework for temporal unsupervised algorithm evaluation, which detects time-series analytics early. In addition, data mining approaches such as correlation analysis [10] have been applied to equipment maintenance. For instance, Perini [11] used the Markov Chain and autoencoder for off-road vehicle maintenance.
Table 1.
 The main anomaly detection-related approaches in predictive maintenance.
Method Studies
CNN [12,13][12][13]
NN-related

(e.g., ANN, autoencoder)		 [11,14,15][11][14][15]
Machine learning [4,5,6,7,8,9,16][4][5][6][7][8][9][16]
Data mining (correlation analysis, Markov Chain) [10,11][10][11]
Mathematics (e.g., fuzzy) [17,18][17][18]
Recently, anomaly detection using images has attracted many researchers [19]. Therefore, this restudy earch applied image data in data analytics for anomaly detection. First, weresearchers used the Publish or Perish software to survey emerging studies’ time trends and disciplinary distribution and related literature were retrieved. Notably, this studyresearch included keywords from top-cited articles to present the current technological trends: “deep learning,” “unsupervised learning”, “semi-unsupervised learning”, “supervised learning”, and “restructure”. These keywords are high-frequency for anomaly detection-related topics concerned with image issues. Then, the retrieved papers were sorted based on their average citations per year and total citations.  Table 2 lists the main methods identified.
Table 2.
 The main anomaly detection approaches from 2017 to 2022 that used images.
Method Studies
Generative adversarial networks (GAN) [20,21][20][21]
Convolutional neural network (CNN) [22,23,24,25][22][23][24][25]
Neural networks (NN)-related [26,27][26][27]
Image process technologies [28,29,30,31,32][28][29][30][31][32]
Mathematics [33,34][33][34]
Most findings have demonstrated that convolutional neural network (CNN) and image process approaches are the core approaches that drive research associated with anomaly detection, especially in the medical, manufacturing, and transportation fields. The result of ourthe literature retrieval also showed that deep learning approaches, such as the ImageNet dataset, cancer diagnosis, and defective product detection, exhibited high accuracies in some cases. As a result, deep learning for anomaly detection has gained massive popularity in anomaly detection-related research. Of these deep learning approaches, CNN can learn directly from data. Previously, Haselmann et al. [22] used a deep CNN to recognize an anomalous surface. In their study, the CNN framework extracted the features of the cell image through a self-learning capability. Similarly, while Xu et al. [23] used a hierarchical CNN to detect anomalous chest X-ray images, Nguyen et al. [25] used a deep CNN to restructure images and detect the anomalous region of magnetic resonance imaging scans. Khan et al. [24] also used a deep CNN to recognize anomalous spectrograms. Accordingly, generative adversarial networks (GANs) are another deep-learning method that uses an ensemble neural network model (e.g., CNN) for automatically discovering and learning the regularities or patterns in input data. GANs are also usually used in data augmentation. In previous studies, while Deecke [20] used the GAN in visual inspection for anomaly detection, Berg et al. [35] used it to identify anomalous contaminated image data. Zhou et al. [21] also used the GAN to identify anomalous retinal optical coherence tomography images. The above studies show that deep learning approaches achieve state-of-the-art performance in image anomaly detection using images. However, for the expected reason, the original image data have noise. For example, Chithirala et al. [36] indicated that image data with noise may influence the deep learning analysis result. Therefore, the original image dataset must be preprocessed. A study reported that while the filters used in image processing can enhance the image or edge features [37], they also effectively remove noise and blurred areas in images. Hence, reducing the noise in an image is the main preprocessing stage before being imported into the deep learning model for training [38]. It has also been reported that while performing image denoising, sufficiently retaining the features in the image is necessary, which is a critical aspect during preprocessing. Some studies have proposed image processing technologies: mathematical methods or neuron networks (NNs) to analyze image data for identifying an anomaly. For example, Zhang et al. [28] proposed a three-stage tensor decomposition and divided it into three steps for detecting anomalies from hyperspectral images. Similarly, while Ayhan et al. [29] proposed a two-step alignment approach for multispectral image anomaly detection, Cohen and Hoshen [39] used correspondences based on a multiresolution feature pyramid in the sub-image detection. In addition, Mishra et al. [40] applied the transformer method in image localization and anomaly detection. However, Mishra et al. [41] proposed a deep reconstruction-based pyramidal approach to exact image features for anomaly detection. Likewise, although Zhuang et al. [30] proposed a robust hyperspectral image denoiser to process its anomaly detection, Müller et al. [27] used NN for feature extraction. Alternatively, another study employed the support vector machine and Gaussian mixture models to recognize anomalous images Müller et al. [27]. Accordingly, while Cozzolino and Verdoliva proposed an autoencoder method for recognizing the anomalous-spliced image region, Li et al. [33] proposed a Gaussian distribution model estimation method to identify the anomalous image descriptors in traffic videos. Furthermore, although Verdoj and Grangetto [31] used the graph Fourier transform to enhance the efficacy of the Reed–Xiaoli detector for medical image anomaly detection, Wang et al. [34] used the discrete probability model and deep autoregressive module for image anomaly detection. Lastly, Vojir et al. [32] also proposed a reconstruction module to identify unknown objects for autonomous driving anomaly detection.​​

References

  1. Kamat, P.; Sugandhi, R. Anomaly detection for predictive maintenance in industry 4.0—A survey. E3S Web Conf. 2020, 170, 02007.
  2. IBM Cloud Education. The Basics of Business Automation. IBM Official Website. Available online: https://www.ibm.com/cloud/blog/basics-of-business-automation (accessed on 12 May 2021).
  3. Stojanovic, L.; Dinic, M.; Stojanovic, N.; Stojadinovic, A. Big-data-driven anomaly detection in industry (4.0): An approach and a case study. In Proceedings of the 2016 IEEE International Conference on Big Data (Big Data), Washington, DC, USA, 5–8 December 2016; pp. 1647–1652.
  4. Winters, P.; Adae, I.; Silipo, R. Anomaly Detection in Predictive Maintenance. In Anomaly Detection with Time Series Analysis; KNIME: Zurich, Switzerland, 2014; pp. 3–9.
  5. Erdmann, M. Unsupervised Anomaly Detection in Sensor Data Used for Predictive Maintenance. Doctoral Dissertation, Ludwig-Maximilians-University, Munchen, Germany, 2018.
  6. Minarini, F. Anomaly Detection Prototype for Log-Based Predictive Maintenance at INFN-CNAF. Master’s Thesis, University of Bologna, Bologna, Italy, 2019.
  7. Alaoui-Belghiti, A.; Chevallier, S.; Monacelli, E. Unsupervised anomaly detection using optimal transport for predictive maintenance. In Proceedings of the International Conference on Artificial Neural Networks, Munich, Germany, 17–19 September 2019; Springer: Cham, Switzerland, 2019; pp. 686–697.
  8. Farbiz, F.; Miaolong, Y.; Yu, Z. A cognitive analytics based approach for machine health monitoring, anomaly detection, and predictive maintenance. In Proceedings of the 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), Kristiansand, Norway, 9–13 November 2020; pp. 1104–1109.
  9. Carrasco, J.; López, D.; Aguilera-Martos, I.; García-Gil, D.; Markova, I.; García-Barzana, M.; Herrera, F. Anomaly detection in predictive maintenance: A new evaluation framework for temporal unsupervised anomaly detection algorithms. Neurocomputing 2021, 462, 440–452.
  10. Zhao, P.; Kurihara, M.; Tanaka, J.; Noda, T.; Chikuma, S.; Suzuki, T. Advanced correlation-based anomaly detection method for predictive maintenance. In Proceedings of the 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), Dallas, TX, USA, 19–21 June 2017; pp. 78–83.
  11. Perini, L. Predictive Maintenance for Off-Road Vehicles Based on Hidden Markov Models and Autoencoders for Trend Anomaly Detection. Doctoral Dissertation, Politecnico di Torino University, Torino, Italy, 2019.
  12. Serradilla, O.; Zugasti, E.; Ramirez de Okariz, J.; Rodriguez, J.; Zurutuza, U. Adaptable and explainable predictive maintenance: Semi-supervised deep learning for anomaly detection and diagnosis in press machine data. Appl. Sci. 2021, 11, 7376.
  13. Davari, N.; Veloso, B.; Ribeiro, R.P.; Pereira, P.M.; Gama, J. Predictive maintenance based on anomaly detection using deep learning for air production unit in the railway industry. In Proceedings of the 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), Porto, Portugal, 6–9 October 2021; pp. 1–10.
  14. De Benedetti, M.; Leonardi, F.; Messina, F.; Santoro, C.; Vasilakos, A. Anomaly detection and predictive maintenance for photovoltaic systems. Neurocomputing 2018, 310, 59–68.
  15. Bose, S.K.; Kar, B.; Roy, M.; Gopalakrishnan, P.K.; Basu, A. ADEPOS: Anomaly detection based power saving for predictive maintenance using edge computing. In Proceedings of the 24th Asia and South Pacific Design Automation Conference, Tokyo, Japan, 21–24 January 2019; pp. 597–602.
  16. Coelho, D.; Costa, D.; Rocha, E.M.; Almeida, D.; Santos, J.P. Predictive maintenance on sensorized stamping presses by time series segmentation, anomaly detection, and classification algorithms. Procedia Comput. Sci. 2022, 200, 1184–1193.
  17. Decker, L.; Leite, D.; Giommi, L.; Bonacorsi, D. Real-time anomaly detection in data centers for log-based predictive maintenance using an evolving fuzzy-rule-based approach. In Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Glasgow, UK, 19–24 July 2020; pp. 1–8.
  18. Decker, L.; Leite, D.; Viola, F.; Bonacorsi, D. Comparison of evolving granular classifiers applied to anomaly detection for predictive maintenance in computing centers. In Proceedings of the 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS), Bari, Italy, 27–29 May 2020; pp. 1–8.
  19. Mohammadi, B.; Fathy, M.; Sabokrou, M. Image/video deep anomaly detection: A survey. arXiv 2021, arXiv:2103.01739.
  20. Deecke, L.; Vandermeulen, R.; Ruff, L.; Mandt, S.; Kloft, M. Image anomaly detection with generative adversarial networks. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Dublin, Ireland, 10–14 September 2018; Springer: Cham, Switzerland; pp. 3–17.
  21. Zhou, K.; Gao, S.; Cheng, J.; Gu, Z.; Fu, H.; Tu, Z.; Liu, J. Sparse-gan: Sparsity-constrained generative adversarial network for anomaly detection in retinal oct image. In Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA, 3–7 April 2020; pp. 1227–1231.
  22. Haselmann, M.; Gruber, D.P.; Tabatabai, P. Anomaly detection using deep learning based image completion. In Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA, 17–20 December 2018; pp. 1237–1242.
  23. Xu, S.; Wu, H.; Bie, R. CXNet-m1: Anomaly detection on chest X-rays with image-based deep learning. IEEE Access 2018, 7, 4466–4477.
  24. Khan, A.S.; Ahmad, Z.; Abdullah, J.; Ahmad, F. A spectrogram image-based network anomaly detection system using deep convolutional neural network. IEEE Access 2021, 9, 87079–87093.
  25. Nguyen, B.; Feldman, A.; Bethapudi, S.; Jennings, A.; Willcocks, C.G. Unsupervised region-based anomaly detection in brain mri with adversarial image inpainting. In Proceedings of the 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), Nice, France, 13–16 April 2021; pp. 1127–1131.
  26. Cozzolino, D.; Verdoliva, L. Single-image splicing localization through autoencoder-based anomaly detection. In Proceedings of the 2016 IEEE International Workshop on Information Forensics and Security (WIFS), Abu Dhabi, United Arab Emirates, 4–7 December 2016; pp. 1–6.
  27. Müller, R.; Ritz, F.; Illium, S.; Linnhoff-Popien, C. Acoustic anomaly detection for machine sounds based on image transfer learning. arXiv 2020, arXiv:2006.03429.
  28. Zhang, X.; Wen, G.; Dai, W. A tensor decomposition-based anomaly detection algorithm for hyperspectral image. IEEE Trans. Geosci. Remote Sens. 2016, 54, 5801–5820.
  29. Ayhan, B.; Dao, M.; Kwan, C.; Chen, H.M.; Bell, J.F.; Kidd, R. A novel utilization of image registration techniques to process mastcam images in mars rover with applications to image fusion, pixel clustering, and anomaly detection. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 4553–4564.
  30. Zhuang, L.; Gao, L.; Zhang, B.; Fu, X.; Bioucas-Dias, J.M. Hyperspectral image denoising and anomaly detection based on low-rank and sparse representations. IEEE Trans. Geosci. Remote Sens. 2020, 60, 1–17.
  31. Verdoja, F.; Grangetto, M. Graph Laplacian for image anomaly detection. Mach. Vis. Appl. 2020, 31, 1–16.
  32. Vojir, T.; Šipka, T.; Aljundi, R.; Chumerin, N.; Reino, D.O.; Matas, J. Road anomaly detection by partial image reconstruction with segmentation coupling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 15651–15660.
  33. Li, Y.; Liu, W.; Huang, Q. Traffic anomaly detection based on image descriptor in videos. Multimed. Tools Appl. 2016, 75, 2487–2505.
  34. Wang, L.; Zhang, D.; Guo, J.; Han, Y. Image anomaly detection using normal data only by latent space resampling. Appl. Sci. 2020, 10, 8660.
  35. Berg, A.; Ahlberg, J.; Felsberg, M. Unsupervised learning of anomaly detection from contaminated image data using simultaneous encoder training. arXiv 2019, arXiv:1905.11034.
  36. Chithirala, N.; Natasha, B.; Rubini, N.; Radhakrishnan, A. Weighted Mean Filter for removal of high density Salt and Pepper noise. In Proceedings of the 2016 3rd International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 22–23 January 2016.
  37. Suhas, S.; Venugopal, C. MRI image preprocessing and noise removal technique using linear and nonlinear filters. In Proceedings of the 2017 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT), Mysuru, India, 15–16 December 2017.
  38. Kusnik, D.; Smolka, B. On the robust technique of mixed Gaussian and impulsive noise reduction in color digital images. In Proceedings of the 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA), Corfu, Greece, 6–8 July 2015.
  39. Cohen, N.; Hoshen, Y. Sub-image anomaly detection with deep pyramid correspondences. arXiv 2020, arXiv:2005.02357.
  40. Mishra, P.; Verk, R.; Fornasier, D.; Piciarelli, C.; Foresti, G.L. VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization. In Proceedings of the 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), Kyoto, Japan, 20–23 June 2021; pp. 1–6.
  41. Mishra, P.; Piciarelli, C.; Foresti, G.L. A neural network for image anomaly detection with deep pyramidal representations and dynamic routing. Int. J. Neural Syst. 2020, 30, 2050060.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback