Encyclopedia
Please note this is an old version of this entry, which may differ significantly from the current revision.
Subjects:
Engineering, Manufacturing
Arc welding is the common method used in traditional welding, which constitutes the majority of total welding production. The traditional manual and manual teaching welding method has problems with high labor costs and limited efficiency when faced with mass production. With the advancement in technology, intelligent welding technology is expected to become a solution to this problem in the future. To achieve the intelligent welding process, modern sensing technology can be employed to effectively simulate the welder’s sensory perception and cognitive abilities.
- sensing technology
- neural networks
- attention mechanisms
1. Introduction
Arc welding is the common method used in traditional welding, which constitutes the majority of total welding production, and is commonly applied in automobile production, shipbuilding, and other fields [1]. Although most precision welding still requires manual labor, the traditional manual and manual teaching welding method has problems with high labor costs and limited efficiency when faced with mass production. Intelligent welding technology is an effective idea proposed to solve this problem, which has become an important research direction in the development of welding technology. Figure 1 shows the various stages in development that arc welding has undergone. During the initial stages, welding was carried out manually, resulting in the welding quality being heavily reliant on the theoretical expertise and practical experience of the technicists. Then, the advancement of robot technology led to the gradual substitution of manual welding with industrial welding robots, representing the second stage in automated welding. Industrial robot welding enables efficient welding in challenging areas that are impossible for human welders [2]. But the majority of welding robots continue to operate using manual teaching methods. In the actual welding process, the errors of equipment, preprocessing process, and environmental factors must be considered, and weld identification and defect detection still rely on manual examination and training [3]. Aimed at these problems, the application of different sensing technologies has made intelligent robotic welding possible [4]. Sensing technologies have been demonstrated to have remarkable effectiveness in recognizing welding environments, monitoring the plasma welding process, and detecting weld defects [5].
Figure 1. Three development stages in arc welding.
The value of sensing technology in the advancement of intelligent welding depends on its ability to simulate the welder’s sensory and cognitive capabilities at an advanced level. Welders can acquire information from the arc and environment by paying close attention to the workpiece and the welding pool throughout the welding process. Afterward, combined with existing experience and theory, the defects in the weld seam are determined. At last, an evaluation of the welding quality is carried out. The investigation of welding sensing technologies can be summarized into three aspects: (1) employing different sensors to acquire a substantial quantity of reliable welding process data; (2) employing efficient signal processing algorithms to obtain suitable signal features; (3) establishing models of the welding tasks and finding a dependable correlation between characteristics and sensing targets.
Sensing technology is the basis of intelligent arc welding. Currently, many sensing technologies have been employed for various welding targets. Figure 2 summarizes the monitoring targets of regularly employed sensing technologies, such as visual, arc, acoustic, spectral, and other nondestructive testing (NDT) sensing signals [6]. Welding dynamics and welding parameters have substantial connection, and welding is a nonlinear, uncertain, and time-varying process. As a result, it is challenging to extract and simulate the characteristic factors throughout the welding process.
Figure 2. An overview of the monitoring targets and sensing technologies.
2. Sensing Technologies for the Welding Process
2.1. Basic Sensing Technologies
2.1.1. Vision Sensing
Vision sensing technology possesses the benefits of simple equipment setup, mature supporting algorithms, and good stability. Therefore, visual sensors are widely used in fields such as weld recognition, weld tracking, molten pool monitoring, and weld quality monitoring [12,13]. As shown in Figure 3, according to the light source properties, the system can be divided into active and passive vision sensing. Active vision sensing technology is usually used to realize weld recognition and dynamic tracking functions during robot welding. It uses a line laser light source to illuminate the weld, obtains the coordinates of the weld center point through an image processing algorithm, and then uses the calibration relationship of the sensor to guide or correct the position of the welding gun [14]. Passive vision sensing technology does not use additional light sources. The advantage is the large amount of image information.
Figure 3. Classification of visual sensing technology.
2.1.2. Acoustic Sensing
Experienced welders can rely on variable sound to judge the welding condition in the welding process. Several studies have found that acoustic data from arcs is essential in determining the quality of welding [16]. The frequency range of 20 Hz to 20 kHz, which falls inside the audible sound band, is frequently employed for monitoring robotic arc welding. Figure 4 shows the mechanism of sound sensing technology; the source of sound in the arc welding process is the alteration of energy in the arc column, which involves the auditory manifestations of arc burning, droplet transfer, and splashing. Arc sound signals can detect even small changes in the metal transmission condition and variations in welding conditions. So, the acoustic sensing has rapidly developed as an essential tool for monitoring welding processes in real time.
Figure 4. The mechanism of sound sensing technology.
2.1.3. Arc Sensing
The current and arc voltage signal are important parameters in the arc welding process. The intensity of the welding current signal can effectively reflect the heat input during the welding process, which can help researchers monitor the penetration status of the molten pool; the arc voltage signal can effectively reflect the arc length and stability during the welding process, connecting with some specific welding defects, such as arc breakage, welding leakage, and other defects [20].
2.1.4. Spectral Sensing
The arc spectrum is a noncontact sensing information from arc emission, which shows good sensitivity to dynamic welding processes and weld defects [22,23]. Welding data are abundant in the arc spectrum. In comparison to other sensing technologies, its advantage lies in its ability to monitor the alterations of elements in the arc atmosphere. Figure 5 shows the arc spectrum mechanism. The line spectrum of metallic and nonmetallic elements is derived from the welding base metal, welding wire, and shielding gas, including aluminum, hydrogen, and argon. Variations in the concentration of these elements can indicate the stability in the welding process and the diffusion of elements, both of which are connected to the formation of porosity defects.
Figure 5. The mechanism of arc spectrum sensing.
2.1.5. Other Sensing Technologies
In addition to the abovementioned technologies, there are also some other sensors, such as infrared, ultrasonic, and radiographic sensing. Infrared sensors and infrared cameras have the ability to record temperature data. One major benefit of these sensors is their resistance to interference from welding noise, making them suitable for monitoring molten pools [26] and detecting defects such as undercuts, humps, and unfused flaws [27]. Ultrasonic sensing [28] and radiography sensing [29] are typical nondestructive testing methods and are currently widely used in weld quality assessment after the welding process.
2.1.6. Multisensor Fusion
It is difficult to manage all types of welding defects with only single-sensing information. So, in the application of welding sensing information, in order to maximize the advantages of different sensors and achieve better monitoring effects, multi-information fusion technology is generally used to coordinate multiple-sensing signals to monitor welding quality and achieve intelligent welding. The performance of multisensor fusion is influenced by various parameters, including the dimensionality of features and fusion techniques, yet it can accomplish functions that are not possible with typical single-source sensing. In recent years, the advancement of neural network technology and the proposal of attention mechanism have also made the fusion of different sensing information more flexible.
2.2. Applications of Welding Sensing Information
2.2.1. Weld Path Recognition
Weld path recognition is an essential initial step of robot welding; the initial point positioning and the spatial weld identification are the two basic research directions. Vision sensing technology is the main sensing technology to realize this application field.
The start welding point positioning can be divided into passive visual detection and active visual detection according to the light source. For an active vision system, the start welding point positioning is usually based on the key point detection algorithm, the geometric correlation, or visual pattern. Various techniques are employed for detecting edges and extracting foreground in weld extraction, such as the Canny operator, Sobel operator, and threshold segmentation, among others. Chen et al. [30] proposed a two-step start welding position locating technique that employs the intersection approach and local corner detection. The effectiveness of this method in accurately determining the 3D position of both straight and curved welds has been demonstrated.
The purpose of spatial weld extraction is to achieve a high-speed spatial weld location and simultaneous quick extraction of the entire weld seam. The crucial technology for this is stereovision. Peng et al. [34] developed a highly effective method for generating robotic welding paths. A new method called the local and global groove feature histogram was introduced as a descriptor for surface deformation in order to precisely locate the weld seam. This method was shown to attain a high level of precision in locating welds.
2.2.2. Weld Seam Tracking
Weld seam tracking is the basis for realizing the robotic intelligent welding process. Its main purpose is to monitor the relative position of the welding gun and the weld seam in real time to ensure the accuracy of the welding trajectory.
The aim of welding deviation monitoring is the welding errors according to the analysis of the sensing information obtained throughout the welding. In recent years, some studies have shown that arc sensing [37] and acoustic sensing [38] signals are correlated with the arc length during the welding process. In terms of arc sensing, Liu [39] et al. introduced a technique called harmonic extraction, which is used in the frequency domain to effectively retain the crucial arc signal information following the application of filtering.
2.2.3. Weld Pool Monitoring
A stable weld pool is an essential part in the arc welding process. It can relate to significant issues with the quality of the welding, such as lack of penetration and fusion. So, the monitoring of the weld pool in real time is important for intelligent arc welding. According to the type and dimension of molten pool information, molten pool monitoring during the welding process can be divided into three categories, including thermal imaging information monitoring, molten pool morphology monitoring, and three-dimensional reconstruction of the molten pool.
The weld pool geometry and temperature information can be monitored by thermal images; its advantage is that it can directly obtain the temperature field distribution in the molten pool during the welding process and accurately monitor it. Subashini et al. [47] proposed a new algorithm to detect the geometric properties of the weld pool from infrared images based on k-means. Then, the ANFIS model received input with the combination of geometric features, peak temperature, and Gaussian thermal area to calculate the penetration status and bead width.
The monitoring of the molten pool morphology mainly relies on passive visual sensing. The detection information includes molten pool morphology, backside molten width, molten pool stability, and monitoring the penetration status of the weld and some common defects (welding leaks, wrong edges, lack of fusion, etc.).
2.2.4. Real-Time Weld Quality Monitoring
Real-time weld quality monitoring is a real-time identification and prediction of welding defects based on sensing information. This technology is an important part of ensuring the quality of welds produced by robot intelligent welding. In recent years, researchers have carried out many studies in this field, and have made great progress in identification accuracy and efficiency of welding defects. Early research mainly simulated welder behavior and conducted feature extraction and analysis, based on the correlation between vision, sound, arc signals and various defects. However, the defects were mainly targeted on the external defects, such as incomplete penetration, burn-through, undercut, and wrong edge. With the advancement in sensing technology and the improvement of weld quality requirements, some sensing information that exceeding the human sensory level was introduced into the weld sensing system, and as the scope of defect detection has become more comprehensive, the real-time monitoring of internal defects such as pores and slag inclusions became possible.
As mentioned above, visual sensing technology can collect rich welding pool information in real time. Through the geometric feature and stability of the molten pool, the penetration status can be monitored and the surface defects of some welds can be predicted. Chen et al. [55] extracted the visual features which contributed to the penetration prediction by a multiangle visual system; the prediction result is shown in Figure 9.
Figure 9. The penetration prediction result of classification and regression [55].
The time-varying properties of arc signals are complicated, and are sensitive to arc disturbance in the welding process. It is widely used in real-time weld quality diagnosis. Xiao et al. [62] used arc voltage variation to research the relation that exists between the natural oscillation frequency and the geometry of the weld pool for both partial penetration and full penetration.
The spectral signal can be analyzed through spectral lines to obtain the changes in welding heat input and arc elements content. Combined with the formation mechanism of various internal defects, the time domain characteristics of each element spectral line in the arc emission spectrum are an important basis for solving difficult-to-predict internal defects such as pores and slag inclusions.
In addition, the development of numerical simulation technology is also an important part of the field of welding quality monitoring. Combined with welding process information, numerical simulation calculations can accurately predict the welding dynamic process including molten pool oscillation, penetration state, etc. It can provide a theoretical basis for the application of sensing information in the field of intelligent welding.
3. Deep Neural Network and Attention Mechanism for Welding Dynamical Process
Sensing technology is the basis for realizing the intelligent welding process. These technologies provide effective information for the initial positioning, path planning, weld tracking, and defect detection in the robot welding process. However, there are differences between sensing information, and the original signals of various sensors are characterized by noise interference and high information redundancy. Therefore, the pre-processing and feature extraction of sensing signals, together with model establishment for different targets, are key issues that need to be researched and solved. Traditional feature extraction and modeling methods are mostly based on theory and numerical calculations to establish mathematical models of welding information and targets. However, the welding process is highly nonlinear and time-varying, and its parameter distribution contains a large number of uncertain factors. The traditional method for obtaining accurate mathematical models through high-dimensional sensing information is often extremely difficult or infeasible. The rapid development of neural network technology has provided new ideas for the processing and modeling process of sensing information. The feature extraction process has been greatly optimized, and the effect of the model has reached a new height.
3.1. Neural Network and Deep Learning
Deep neural network technology is a branch of traditional machine learning technology. It imitates the structure and function of the human brain and uses multilayer neural networks connected to each other to learn and analyze data. Deep learning (DL) is an advancement of traditional shallow neural networks, marked by a significant increase in the number of hidden layers. Hidden layers transform the input data into higher-dimensional spaces, allowing for analysis of the input data from multiple viewpoints. Increasing the number of hidden layers enhances the probability of uncovering underlying patterns in the data. In recent years, DL algorithms have become increasingly widely used in computer vision, speech recognition, natural language processing, and feedback control, and have created many network structures and algorithms, including feed-forward neural networks (FNN), convolution neural network (CNN), recurrent neural network (RNN), restricted Boltzmann machine (RBM), etc. [72] In the field of welding sensing, deep learning algorithms are mainly used in extracting characteristic information from sensor data and relating the characteristic information to target information of interest through learning models.
3.1.1. Network Structure of Deep Learning
Among the deep learning network structures, CNN and RNN are the two most typical and widely used. Most network structures in the field of welding sensing are derived from these two. Figure 13 shows the typical neural network structure and classification.
Figure 13. Neural network structure and classification.
In 1980, Fukushima introduced the initial convolutional neural network (CNN), which drew inspiration from the architectural structures of animal visual cortexes for the purpose of visual perception [73]. LeCun et al. [74] developed the original contemporary convolutional neural network (CNN), known as LeNet-5, in 1998, and effectively employed it for the recognition of handwritten digits. Afterward, many new networks were derived to address the shortcomings of traditional neural network structures, such as AlexNet, VGGNet, GoogLeNet, GAN, and ResNet. Convolutional layers, pooling layers, and fully connected layers, as three types of layers, form the structure of CNNs. The convolution operations are used to extract feature maps from input images or grid-like data. This is accomplished by convolving the feature maps from the preceding layer with small-size kernels that shift over the data. A kernel refers to a collection of weights that can be learned and are not randomly assigned, but rather updated repeatedly during the process of learning. The initial step requires defining only the kernel size. The appropriate selection of kernel size should be based on the proportional dimensions of the item (or region) of interest in relation to the size of the image. Due to advancements in deep learning, it is preferable to have a small kernel size in order to effectively capture local information. The processing of vision-sensing information in the welding process is a typical machine vision field. Therefore, research on feature extraction and target detection is mostly based on the CNN network structure.
3.1.2. Deep Learning in Welding
The proposal and development in the concept of deep learning has greatly promoted the progress of intelligent welding technology based on multi-information sensing. Traditional feature extraction and modeling methods have certain limitations when faced with complex sensing information and highly nonlinear welding processes. Neural network technology solves this problem well in an end-to-end method.
In terms of CNN, Chen et al. [77] proposed a multiscale activation module that employs convolution kernels of varying structure to enhance the contextual information of the spectrum. This approach demonstrated remarkable predictive accuracy for determining the composition of wires and the rate of gas flow. Zhang et al. [78] proposed a deep learning algorithm based on CNN for the detection of welding defects in three different conditions during high-power disk laser welding.
RNN and CNN have their own advantages under different application conditions. With the continuous development in sensing technology and the continuous improvement in welding quality requirements, the application of deep neural network technology is not limited to a single network structure, but is more inclined to the fusion of multiple network structures, when facing the same goal, divides the task goals into blocks, and uses the best network structure for each block to maximize the advantages of each network model.
3.2. Attention Mechanism
Attention is a complicated mental process that is essential for humans to survive [86]. An essential characteristic of perception is that people typically do not handle complete information simultaneously in its full extent. Instead, people have a tendency to focus on some particular piece of information, while disregarding other observable information simultaneously. This is a method for humans to efficiently choose valuable information from a large amount of information using limited brain power. The development of the attention mechanism significantly enhances the effectiveness and precision of perceptual information processing [87]. It can be found that sensing information, as a computer perception method, can also optimize its calculation process by imitating the human attention mechanism. Recently, the attention mechanism has been employed as a method for allocating resources to address the issue of information overload. When processing power is restricted, it has the ability to prioritize and handle crucial information using available resources [88]. As research progresses, the attention mechanism has been a popular component in neural networks and has been utilized in a wide range of tasks, including picture caption generation, machine translation, action identification, and graph analysis. The attention mechanism not only enhances performance, but it can also serve as a tool to elucidate perplexing neural network architectural behavior [89].
3.2.1. Theory of the Attention Mechanism
The implementation procedure for the attention mechanism involves two parts: firstly, calculating the attention distribution for the input data and secondly, computing the context vector based on the attention distribution.
Figure 15 demonstrates that during computation of the attention distribution, the neural network initially encodes the feature of the source data as K, which is referred to as a key. K can be represented in different forms depending on the individual tasks and neural architectures. Furthermore, it is typically important to incorporate a task-specific representation vector q, commonly referred to as the query. The representation of q might vary depending on the individual activities, either as vectors or as a matrix.
Figure 15. The architecture of the unified attention model.
The neural network calculates the correlation between K and q using the score function f, resulting in the energy score e. This score indicates the significance of queries in determining the next output in relation to the keys. The score function f is an essential component in the attention model as it determines the matching or combination of keys and queries. Additive attention (such as the alignment model in RNN search) [90] and the computationally less expensive multiplicative (dot-product) attention [91] are the two most widely utilized attention methods. In the actual application, it is necessary to select an appropriate score function based on the type of the input signal, the network, and the processing task.
3.2.2. Classification of Attention Mechanisms
The principle of attention models is the same, but in recent years, researchers have made some modifications and improvements to the attention mechanisms in order to better adapt them to specific tasks.
The soft attention mechanism involves the utilization of a weighted average of all keys to construct the context vector. The soft attention mechanism allows for differentiability of the attention module concerning the inputs, enabling the entire system to be trained using traditional back-propagation methods. In the hard attention mechanism, the context vector is calculated based on randomly selected keys. The important difference between the two forms of attention is whether the attention weights are differentiable. In contrast to the soft attention model, the hard attention model is more computationally efficient as it avoids the need to calculate attention weights for all items at each time step.
3.2.3. Attention Mechanism in Welding Sensing
The essence of welding sensing technology is the processing and calculation of visual, sound, spectrum, and other sensing signals. At present, neural network technology is an effective means to realize the calculation process and is widely used in various welding sensing tasks. The attention mechanism has good compatibility with neural network technology. When applied to the field of welding, it can imitate experienced welders to filter sensing information based on importance. On the other hand, further research and discussion on the weld mechanism can be carried out based on the attention weight information obtained from training.
Most of the welding sensing information needs to be combined with the physical mechanism of the welding process to perform corresponding pre-processing operations, and the training of related models mostly uses neural network learning methods; therefore, the soft attention mechanism is suitable for most welding sensing tasks. Among them, the channel attention mechanism and the spatial attention mechanism are the two most widely used forms. The channel attention mechanism mainly focuses on sequence information. Attention weights are attached to different feature channels. It generally requires pre-feature extraction and convolution operations. It has the widest application range and can be applied to various transmission methods such as arc, sound, spectrum, and image sense field.
The channel attention mechanism and the spatial attention mechanism have their own advantages in different aspects. Therefore, in recent years, researchers have connected the two attention mechanisms and worked together for deep learning of welding sensing information, achieving good results. Xiao et al. [99] proposed a multiscale convolution assemble block for spot welding, in which a fused attention block is designed to calibrate the spatial and channel information of welding spot feature maps. The results of classification experiments demonstrated that the proposed strategies are efficient and the accuracy reaches to 95.2%.
At present, the applications of attention mechanism in the field of welding sensing are developing rapidly, but the existing technology is mainly aimed at sound and image information, the application of attention mechanism in spectrum and arc still needs to be developed. With the expansion of application scope and research depth, the attention mechanism is expected to become crucial for further progress in welding sensing technology.
This entry is adapted from the peer-reviewed paper 10.3390/jmmp8010022
This entry is offline, you can click here to edit this entry!
