In the field of railway transportation, fault detection for key components in the braking system of freight trains is critical for ensuring railway transportation safety [1] since the abnormal state of these components can result in serious consequences. Therefore, it is very necessary to detect the abnormal state of these components of freight trains during transportation so as to deal with these anomalies in time to ensure the safety of freight trains [2]. The traditional method of fault detection is mainly carried out by manual work. The train inspection personnel determine whether each component is in the abnormal state by means of ‘‘touch, look, and listen’’ [3]. The efficiency of manual detection depends entirely on the working state of the inspectors, and the subjectivity is pretty strong [4]. In order to improve the efficiency and accuracy of fault detection, the methods based on image processing and machine vision are gradually applied in the field of fault detection for key components in the braking system of freight trains [5].

As one of the key components in the braking system, the angle cock is the switch of the train ventilation duct, which realizes the braking through the air transmission between carriages, so that the train can achieve the purpose of regulating speed or stopping [6]. If the state of angle cock is abnormal, the train will be unable to brake normally, which will lead to overrunning, rear-end collision, etc. For an angle cock, there are three steps in the process of changing it from the closed state to the fully open state, namely lifting the handle of angle cock upward, rotating its handle to the left, and pressing its handle downward. Once the train staff misses the process of pressing its handle down, the angle cock will be in another state other than the closed and fully open state, which is called the misalignment state. If the angle cock is in the fully open state, the air in the ventilation duct flows normally. If the angle cock is in the closed state, the air in the ventilation duct cannot flow properly. But if the angle cock is in the misalignment state, the air will leak into the external environment during transmission, which leads to the reduction of air in the ventilation duct and prevents normal air circulation. Therefore, the misalignment state, along with the closed state, belongs to the abnormal state. In the fault detection of angle cocks, it is necessary to detect not only the closed state, but also the misalignment state. The existing methods equate the non-closed state (which includes the fully open state and the misalignment state) to the fully open state and implement the fully open and closed state detection. For example, the CNN-based detector called Light FTI-FDet model, proposed by Zhang et al. [7], can only implement the fully open and closed state detection for angle cocks, though it can achieve multi-fault detection for freight trains. The real-time and accurate fault inspection approach proposed by Zhou et al. [8] is only used to detect the missing handle of an angle cock. Apparently, the misalignment state is ignored or misclassified as normal. Therefore, it is essential to further identify the misalignment state from the non-closed state.

The whole process of this method is divided into two steps: localization and detection. In the localization step, the camera is installed in the wilderness. Therefore, the image quality will be affected by environmental factors such as weather and light [9]. Since the camera installation location is not fixed, the angle cock cannot be localized by the relative location relationship between itself and other parts [10].

2. Coarse-to-Fine Localization for Detecting Misalignment State of Angle Cocks

In the fault detection of freight trains, most of the studies based on image processing and machine vision extract relevant image features to determine whether the component is in the abnormal state. The common image features include two categories: human-designed features and deep features. The basic image features such as shape, texture, and fusion features belong to the human-designed features. And the features obtained by training with deep learning models belong to the deep features ^[11][12]. For the fault detection of various components, suitable feature representations need to be selected according to their own characteristics, and specific fault judgment guidelines should be used ^[12][13]. For human-designed features, Liu et al. ^[12][13] used sparse histograms of oriented gradients (SHOG) combined with a SVM model to identify the fault of bogie block key with an accuracy of 99.86%. Liu et al. ^[13][14] proposed a method that combines gradient coding co-occurrence matrix (GCCM) and AdaBoost to detect the state of bogie block key with an accuracy above 99.6%. Li et al. ^[14][15] used the local binary patterns (LBP) and a SVM model to judge whether the bolts were in the normal state. The accuracy and false alarm rate were 100% and 8.1%, respectively. Liu et al. ^[15][16] achieved the state detection of fastening bolts through gradient orientation co-occurrence matrix (GOCM) combined with a SVM model, whose accuracy reached 99.91%. Cha et al. ^[16][17] used Hough transform and the height of bolts to represent two states of looseness and tightness. Following that, a linear support vector (LSVM) was used to detect which state the bolts were in, and the accuracy reached 95.45%. Zheng et al. ^[17][18] combined the gradient magnitude and the histogram of gradients in six directions to form the multi-dimensional features (MDF) to locate the coupler yoke with a SVM model. Following that, Haar-like features and AdaBoost decision trees were selected to detect the loss of bolts with an accuracy of 98.60% and a false alarm rate of 4.1%. The human-designed features require different feature extraction algorithms for different targets. And it is difficult to find the most suitable features to characterize the targets, which leads to being unable to detect multiple targets at the same time. For deep features, since the deep learning models can adaptively extract optimal features and detect multiple targets at the same time, they are widely used in the field of fault detection. Sun et al. ^[18][19] used convolutional neural networks to achieve coarse localization of end bolts and side keys. Next, the precise localization was accomplished through the prior information, geometric and spatial location relationships, and another neural network was used to achieve fault detection. The accuracy of fault detection for these two components was 97.50% (when the bolts of shaft bolts are missing), 92.5% (when the bolts of shaft bolts are loose), 100% (when the shaft bolts are missing), and 100% (when the side frame keys are missing). Ye et al. ^[19][20] proposed a multi-feature fusion network (MFF-net), in which the multi-branch dilated convolution module (MDCM), squeeze, and excitation block (SEB) were added to achieve fault detection of side bolts, bottom bolts, and the retaining key. Finally, the mean average precision reached 92.55%, 97.18%, and 97.41%, respectively. Zhang et al. [7] proposed a CNN-based Light FTI-FDet, using multi-region proposal network (MRPN) and model reduction scheme to achieve fault detection of angle cock, bogie block key, dust collector, cut-out cock, and fastening bolt. Finally, the accuracy reached 100%, 99.86%, 99.53%, 96.24%, and 100%, respectively. Ye et al. ^[20][21] used a multi-mode aggregation feature enhanced network (MAFENet) based on a single-stage detector (SSD) to achieve detection of side bolts, bottom bolts, and the retaining key. Finally, the mean average precision (mAP) reached 97.51%, 97.88%, and 100%, respectively. The deep features require a large amount of data for training, which can extract deep and abstract features of the target. The detection performance is less affected by the image environment factors, while the drawback is complex and computationally intensive.