Defect Synthesis for Automated Visual Inspection: Comparison
Please note this is a comparison between Version 1 by Jun-Geol Baek and Version 2 by Camila Xu.

Defect inspection, which detects defects in real-time and classifies defect types, is one of the key technologies required for smart factory implementation. Defect detection on steel surfaces is an important task to ensure the quality of industrial production. To build an automated visual inspection (AVI) and achieve smartization of steel manufacturing, detecting defects in products in real-time and accurately diagnosing the quality of products are essential elements. 

  • automated visual inspection
  • internet of things
  • generative adversarial networks

1. Introduction

In Industry 4.0, internet of things (IoT) applications are becoming important. IoT connects the physical and digital worlds, enabling smart factory development through faster communication and better analytics [1]. In general, a smart factory is one in which all internal elements are organically connected and operated intelligently based on advanced information and communication technology (ICT). Product quality must be measured in real-time to manufacture products at minimal cost and time. There is an increasing demand for steel products with better surface and shape qualities [2]. The end product of the manufacturing process is directly related to economic factors as it affects productivity. IoT applications in the steel industry can make a variety of industries more efficient and flexible, thereby increasing their productivity and yield [3][4][3,4].
Defects are physical and chemical failures caused by problems in the manufacturing process, facility, or manufacturing environment. Steel is manufactured through various processes such as rolling and forging. During this process, defects such as crazing, inclusions, pitted surfaces, rolled-in scales, and scratches occur, as shown in Figure 1 [5].
Figure 1.
 Types of defects in the steel manufacturing process [6].
Defect inspection, which detects defects in real-time and classifies defect types, is one of the key technologies required for smart factory implementation [7][8][7,8]. Defect detection on steel surfaces is an important task to ensure the quality of industrial production. Defect detection on a steel surface involves three preliminary steps, as shown in Figure 2. The first step is inspection, in which defects on the steel surface are detected by inspection tools [9]. The second step is review, in which images of the detected defects are captured by a specific tool. The third step is the detection and classification of defect types based on the captured images. Steel-surface defect detection processes allow engineers to perform cause analysis and defect control. However, visual inspection relies heavily on the experience and abilities of individual engineers. Additionally, this process is usually performed manually in the industry, making it unreliable and time-consuming. Therefore, automated visual inspection (AVI) targeting the surface quality has emerged as a standard configuration for steel manufacturing mills to improve product quality and promote production efficiency [10]. AVI, which performs classification through image-based algorithms, is not only widely applied to the steel manufacturing process but also to glass, fiber, and semiconductor production processes [11].
Figure 2.
 Steel manufacturing process and defect detection steps [6].
Although a convolutional neural network (CNN)-based AVI model exhibits excellent classification performance for numerous defect types, it has two practical problems in the steel manufacturing process. First, the frequency of defect data occurrence is extremely low, and very little data can be used for the development of a deep learning model [12]. In general, sufficient training data for both defect and normal classes are required to improve the classification performance of deep learning models [13]. However, in the actual industry, the quantity of defective data is minimal compared to that of normal data. When performing AVI with only data collected from the industry, data imbalance issues can result in lower learning rates for defect types and poor performance. Therefore, it is necessary to balance the normal and defective classes. Class imbalance refers to a substantial proportional difference between the classes in the total dataset. When the class distribution is unbalanced, the model is trained with a bias toward the majority class, classifying the class with a large amount of data; however, the opposite is true for the minority class. Furthermore, an imbalanced class distribution can lead to serious type II errors. Therefore, preprocessing for class imbalance is essential for improving the overall classification performance in defect detection.
Second, the steel defect data consisted of defects of various sizes. Large defects can be easily generated using a simple generator when a generative model is used to solve the imbalance problem. However, the generation of small-sized defects is significantly influenced by the type of generative model used [12]. In particular, circumstances such as the cold rolling process, where the end product is 2 m wide and the size of the defect is approximately 0.2 mm, require a sophisticated classifier [14].

2. Defect Synthesis for Automated Visual Inspection

In the Introduction, two problems that need to be solved in this researchtudy are discussed. This section addresses related work on the class imbalance problem. In particular, previous studies conducted on steel manufacturing are explored. Figure 3 shows the data imbalance of the Severstal dataset in a visual context [15][18]. The defect data consisted of 53.04% (5680 EA) of the total data, and the proportions of each class were as follows:12.64% (718 EA) for class 1: crazing, 3.48% (198 EA) for class 2: rolled-in scale, 72.59% (4123 EA) for class 3: pitted surface and scratch, and 11.29% (641 EA) for class 4: inclusion. In this researchtudy, the Severstal dataset was sampled and used to create a class imbalance problem. Sampling was performed only in the areas where the defect was present. Section 4 describes Tthe sampling method used can be found in original textin this study.
Figure 3.
 Imbalance ratio of steel defect images in the Severstal dataset.
The numerous solutions proposed to solve class imbalance problems in AVI can be divided into two methods: correcting the model itself and directly processing data [16][19]. In the former, the data instances of different classes are treated differently in a manner similar to active learning or kernel-based methods. In the latter, the direct processing of data utilizes methods such as sampling or data generation to directly control the number of instances.
Sampling is a method used to correct the bias between classes in data with an overwhelmingly small proportion of abnormal data compared with normal data. Representative methods for dealing with class imbalance include oversampling and undersampling. Oversampling is a method for creating new data of the minority class to even the class ratio, whereas undersampling is a method for removing existing data of the majority class to match the ratio. Because undersampling reduces the amount of sample data from the majority class, it has the advantage of reducing the model training time. However, it can also distort data features by removing crucial information. Regarding oversampling, the risk of data distortion is relatively small because it creates new data while preserving original data. The oversampling methods mainly used for AVI include random oversampling, synthetic minority oversampling technique (SMOTE) [17][20], and adaptive synthetic sampling approach (ADASYN) [18][21]. Random oversampling increases the amount of minority class data by randomly selecting and replicating a sample from a minority class. SMOTE synthesizes data by selecting random data belonging to a minority class and randomly selecting the closest top k number of data. ADASYN is a method of adaptively synthesizing k data from marginal minority data according to the number of majority classes after calculating the ratio of the data of a majority class. However, this oversampling method for image data generates low-resolution images.
To simplify this problem, researchwers use a technique to handle the raw image. This method is called data augmentation and has been commonly used as a model regularization technique in recent studies [19][22]. Some common augmentation methods include flipping an image vertically or horizontally, shifting the image vertically or horizontally, and slightly rotating or zooming it. This method helps the training model be robust to small changes in the image. However, simple geometric transformations do not significantly change the image characteristics, making it impossible to identify additional features.
Among data generation methods, GAN is an algorithm of great interest [20][23]. The GAN generates data based on the distribution and exhibits excellent performance in image generation. Various GANs have been studied previously. A deep convolutional GAN (DCGAN) was used for ball-bearing failure detection [21][24]. In addition, a wafer defect image was adaptively generated using conditional GAN (CGAN) [22][25]. The progressive growing GAN (PGGAN) increased the model training speed by gradually increasing the generator and discriminator and producing a high-quality image [23][26]. In addition, to address shortcomings such as the vanishing gradient or mode collapse of a GAN, Wasserstein GAN (WGAN) was proposed [24][27].
Liu et al. [25][28] proposed a GAN-based one-class classification method for detecting strip steel surface defects. Their model achieved 94% good test results on images provided by the Handan Iron and Steel Plant. Lai et al. [26][29] proposed a new detection method using a GAN and statistical-based representation learning mechanism. This method achieved an accuracy of 93.75% on the solar panel dataset. Akhyar et al. [27][30] proposed a method for generating more detailed contours in the original steel image. The method achieves better performance and effectiveness in terms of processing time compared to the original method.
The AVI is critical for effective and efficient maintenance, repair, and operation in advanced manufacturing. However, AVI is often constrained by the lack of defect samples [28][31].
Video Production Service