Federated Few-Shot Learning-Based Machinery Fault Diagnosis: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Rita Xu and Version 1 by Peng Zhao.

In this  paper, we

Researchers

present a novel federated few-shot learning-based machinery fault-diagnosis scheme, named FedCDAE-MN, which utilizes a combination of convolutional denoising auto-encoder and feature-space metric learning across multiple domains.

  • fault diagnosis
  • few-shot learning
  • federated learning

1. Introduction

Mechanical fault diagnosis is of paramount importance in modern industrial contexts as it significantly contributes to enhancing equipment reliability, operational safety, and cost reduction. The emergence of the Industrial Internet of Things (IIoT), big data, and AI has propelled the field of fault diagnosis, leading to notable advancements [1,2][1][2]. These advancements have demonstrated their effectiveness in utilizing condition-monitoring data. As a result, extensive research efforts have been conducted to employ conventional machine-learning techniques [3], such as Support Vector Machines (SVM) [4], k-Nearest Neighbors (KNN) [5], decision trees [6], etc. However, the diagnosing accuracy of these methods often relies on the expertise of specialists who manually select relevant features.
In light of the above limitations, deep-learning (DL) techniques have emerged as a prominent approach, offering the potential for end-to-end learning and the elimination of manual feature selection [7]. However, industrial equipment generally operates under normal conditions, resulting in a limited availability of fault samples. Unfortunately, the conventional deep-learning approach requires a substantial amount of labeled data to achieve a significant level of accuracy. Consequently, numerous recent studies have concentrated on few-shot fault diagnosis [8,9][8][9]. Furthermore, due to the complex and variable working conditions of the equipment, the results of fault diagnosis vary across specific industrial scenarios, which divides fault samples into different domains based on their feature distribution and category space. To address the challenges posed by limited data availability and domain diversity, few-shot cross-domain fault diagnosis emerges as a compelling approach, often referred to as few-shot transfer learning [10,11][10][11]. This technique leverages knowledge from the source domain to the target domain by training a network using data from the source domain on one operational condition and subsequently fine-tuning it with the data from the target domain on different working conditions. This approach has exhibited significant potential to greatly enhance the efficacy of fault diagnosis within the target domain.
In recent years, a few scholars have embarked on the investigation of the few-shot cross-domain fault-diagnosis approach. For instance, Lin et al. [12] introduced a novel cross-domain fault-diagnosis method for bearings, which relies on an enhanced semi-supervised meta-learning technique. Wu et al. [11] presented a few-shot transfer learning method that leverages meta-learning for the diagnosis of few-shot samples under varying conditions. Feng et al. [10] systematically summarized the application of deep meta-learning in the context of few-shot cross-domain fault diagnosis. However, the existing research on few-shot cross-domain fault diagnosis has primarily focused on transferring knowledge from a single source domain to the target domain. Nevertheless, several crucial issues remain unresolved: (1) Practical scenarios often still involve a shortage of all types of fault samples within a single source domain, making the utilization of data from multiple source domains a potentially effective solution. (2) Conducting few-shot machinery fault diagnosis across multiple source domains introduces several new challenges. First, the distinct source domains can be considered to be independent factories or data holders, who are reluctant to disclose or share their private data due to concerns related to industry competition, user privacy, and data security [13,14][13][14]. Second, fault samples acquired from multiple source domains may exhibit diverse distributions due to disparate working conditions or industrial scenarios [15], therefore further complicating the complexities associated with few-shot fault diagnosis.
Figure 1 illustrates the basic framework of ourthe proposal. In particular, weresearchers consider two primary scenarios: (1) a single source domain to target domain scenario, where the working conditions remain the same, but the fault types vary, and (2) multi-source domain scenarios, where the source domains exhibit varying working conditions while the fault type remains consistent. First, to enhance the model’s generalization performance under diverse conditions, weresearchers propose a few-shot learning method that relies on a convolutional denoising auto-encoder and feature-space metric learning. By effectively extracting fault features using the convolutional denoising auto-encoder, weresearchers improve the model’s domain generalization, robustness, and noise reduction capabilities. Furthermore, the metric embedding module encodes the extracted features, and the meta-learning training method exploits sample similarity information. Importantly, the entire metric learning process operates within the feature space, allowing for effective utilization of label information from the source domain. In addition, to address privacy concerns, the proposed model is developed within the federated learning framework, where individual factories are treated as distinct participants, representing multiple source domains in the few-shot fault diagnoses.
Figure 1. Federated few-shot learning for machinery fault diagnosis across multiple domains.

2. Deep Learning for Fault Diagnosis

Deep learning enables end-to-end training, eliminating the need for manual feature engineering by domain experts. Consequently, numerous researchers have adopted deep-learning approaches to effectively diagnose faults. For instance, Wang et al. [16] successfully combine variational mode decomposition (VMD) with a one-dimensional convolutional network to diagnose bearing faults. VMD effectively removes random noise from the original signal and contributes to feature enhancement. Gao et al. [17] employs a deep belief network to accurately identify bearing defects. The network is initially pre-trained using minimal batch random gradient descent and subsequently fine-tuned using conjugate gradient descent and backpropagation. Additionally, Zhang et al. [18] proposes a method that leverages tiny convolution kernels in the second layer for multi-layer nonlinear mapping and larger kernels in the primary convolutional layer for feature extraction and noise reduction. An et al. [19] suggest the utilization of a long-term and short-term memory network for diagnosing faults in time-varying working condition data. The loss function incorporates the outputs from each time step, and dropout regularization is employed to enhance training efficiency and diagnostic capability. Wang et al. [20] also employs long-term and short-term memory networks for bearing fault diagnosis, integrating residual modules with long-term and short-term memory to leverage both deep local features and global time series features of bearing vibration signals.
Although deep learning has demonstrated its effectiveness in fault diagnosis, it heavily relies on a substantial volume of labeled data. However, due to security considerations, machines may halt immediately after a fault occurs, leading to an insufficient number of fault samples available for training deep-learning models.

3. Few-Shot Learning for Fault Diagnosis

To tackle the challenge of limited fault samples, recent research efforts have centered around few-shot fault-diagnosis techniques, which are generally categorized into three main approaches: data-based, algorithm-based, and model-based. (1) Data-based methods concentrate on improving the training dataset by employing data augmentation strategies, including data generation [21[21][22],22], oversampling [23], and reweighting [24]. (2) Model-based methods constrain the hypothesis space through prior knowledge, such as embedded learning [25,26][25][26] and generative adversarial modeling [27]. (3) Algorithm-based methods primarily utilize prior knowledge to modify the search strategy within the hypothesis space. This category can be further divided into two subcategories: methods that provide good initialization parameters and those that directly train an optimizer to output the search step. Fine-tuning is an effective means of providing good initialization parameters [28], while Model-Agnostic Meta-Learning (MAML) represents a framework for directly training an optimizer to determine the search step. For instance, Wu et al. [11] introduced a few-shot transfer learning approach based on MAML for diagnosing faults in variable conditions. The effectiveness of the proposed method is verified based on a unified one-dimensional convolution network on three public datasets. Wang et al. [29] propose a feature-space metric-based meta-learning model (FSM3) to overcome the challenge of the few-shot machinery fault diagnosis under multiple limited data conditions. FSM3 exploits both the attribute information from individual samples and the similarity information from sample groups. In contrast to FSM3, the model in [30] is trained directly in the data space rather than the feature space. Notably, the approach presented in this study falls within the framework of MAML.
The aforementioned methods primarily concentrate on addressing the issue of fault diagnosis with limited samples within a single source domain. Although these methods exhibit satisfactory performance within the single source domain scenario, their effectiveness in accommodating multiple source domains requires further validation. Additionally, there is a lack of adequate attention paid to maximizing the utilization of fault information from each source domain to acquire additional prior knowledge and enhance the generalization capability of models in a multi-source domain scenario. Furthermore, most research efforts often overlook considerations related to user privacy and data security.

4. Federated Learning for Fault Diagnosis

To address concerns related to data dispersion and privacy protection, federated learning has gained attention from scholars as a promising approach for fault diagnosis in recent years. For instance, Wang et al. [31] introduced a federated adversarial domain generalization network for mechanical fault diagnosis. This model achieves collaborative training between a central server and multiple clients, establishing a global fault-diagnosis model while ensuring data privacy. Zhao et al. [32] proposed a multi-source domain adaptive method that combines transfer learning and federated learning. By leveraging user data while preserving data privacy, this approach achieves accurate identification performance. Chen et al. [33] presented a federated transfer learning framework based on a weighted joint average of differences. This framework effectively coordinates well-trained global diagnostic models with privacy protection. Additionally, they developed a dynamic weighted average algorithm using the maximum mean difference (MMD) and integrated the updated local model using an automatically learned weight. Geng et al. [34] proposed the FA-FedAVG algorithm, which optimizes the model weighting strategy to enhance the influence of high-quality local models. They also introduced a model aggregation strategy based on poor accuracy to reduce iteration count and accelerate convergence during training. Moreover, Chen et al. [35] proposed a joint training method that incorporates meta-learning into a federated learning framework, aiming to enhance the performance of the global meta-learner.
Currently, the fault diagnosis with federated learning is typically accomplished through a combination of local training and cloud parameter aggregation. Building upon this foundation, certain methods mainly concentrate on assessing the distinct contributions of individual participants to the global model. Nevertheless, opportunities for optimization persist, offering the potential to investigate the allocation of varying weights to participants. This exploration could enhance the precision and personalization of model training.

References

  1. Chi, Y.; Dong, Y.; Wang, Z.J.; Yu, F.R.; Leung, V.C. Knowledge-based fault diagnosis in industrial Internet of Things: A survey. IEEE Internet Things J. 2022, 9, 12886–12900.
  2. Ur Rehman, M.H.; Yaqoob, I.; Salah, K.; Imran, M.; Jayaraman, P.P.; Perera, C. The role of big data analytics in industrial Internet of Things. Future Gener. Comput. Syst. 2019, 99, 247–259.
  3. Martin-Diaz, I.; Morinigo-Sotelo, D.; Duque-Perez, O.; Romero-Troncoso, R.J. An experimental comparative evaluation of machine learning techniques for motor fault diagnosis under various operating conditions. IEEE Trans. Ind. Appl. 2018, 54, 2215–2224.
  4. Hu, Q.; He, Z.; Zhang, Z.; Zi, Y. Fault diagnosis of rotating machinery based on improved wavelet package transform and SVMs ensemble. Mech. Syst. Signal Process. 2007, 21, 688–705.
  5. He, Q.P.; Wang, J. Fault detection using the k-nearest neighbor rule for semiconductor manufacturing processes. IEEE Trans. Semicond. Manuf. 2007, 20, 345–354.
  6. Sun, W.; Chen, J.; Li, J. Decision tree and PCA-based fault diagnosis of rotating machinery. Mech. Syst. Signal Process. 2007, 21, 1300–1317.
  7. Zhang, S.; Zhang, S.; Wang, B.; Habetler, T.G. Deep learning algorithms for bearing fault diagnostics—A comprehensive review. IEEE Access 2020, 8, 29857–29881.
  8. Zhang, T.; Chen, J.; Li, F.; Zhang, K.; Lv, H.; He, S.; Xu, E. Intelligent fault diagnosis of machines with small & imbalanced data: A state-of-the-art review and possible extensions. ISA Trans. 2022, 119, 152–171.
  9. Zhang, A.; Li, S.; Cui, Y.; Yang, W.; Dong, R.; Hu, J. Limited data rolling bearing fault diagnosis with few-shot learning. IEEE Access 2019, 7, 110895–110904.
  10. Feng, Y.; Chen, J.; Xie, J.; Zhang, T.; Lv, H.; Pan, T. Meta-learning as a promising approach for few-shot cross-domain fault diagnosis: Algorithms, applications, and prospects. Knowl. Based Syst. 2022, 235, 107646.
  11. Wu, J.; Zhao, Z.; Sun, C.; Yan, R.; Chen, X. Few-shot transfer learning for intelligent fault diagnosis of machine. Measurement 2020, 166, 108202.
  12. Lin, J.; Shao, H.; Min, Z.; Luo, J.; Xiao, Y.; Yan, S.; Zhou, J. Cross-domain fault diagnosis of bearing using improved semi-supervised meta-learning towards interference of out-of-distribution samples. Knowl. Based Syst. 2022, 252, 109493.
  13. Serror, M.; Hack, S.; Henze, M.; Schuba, M.; Wehrle, K. Challenges and opportunities in securing the industrial internet of things. IEEE Trans. Ind. Inf. 2020, 17, 2985–2996.
  14. Gebremichael, T.; Ledwaba, L.P.; Eldefrawy, M.H.; Hancke, G.P.; Pereira, N.; Gidlund, M.; Akerberg, J. Security and privacy in the industrial internet of things: Current standards and future challenges. IEEE Access 2020, 8, 152351–152366.
  15. Wang, R.; Huang, W.; Wang, J.; Shen, C.; Zhu, Z. Multisource domain feature adaptation network for bearing fault diagnosis under time-varying working conditions. IEEE Trans. Instrum. Meas. 2022, 71, 1–10.
  16. Wang, Q.; Yang, C.; Wan, H.; Deng, D.; Nandi, A.K. Bearing fault diagnosis based on optimized variational mode decomposition and 1D convolutional neural networks. Meas. Sci. Technol. 2021, 32, 104007.
  17. Gao, S.; Xu, L.; Zhang, Y.; Pei, Z. Rolling bearing fault diagnosis based on intelligent optimized self-adaptive deep belief network. Meas. Sci. Technol. 2020, 31, 055009.
  18. Zhang, W.; Peng, G.; Li, C.; Chen, Y.; Zhang, Z. A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals. Sensors 2017, 17, 425.
  19. An, Z.; Li, S.; Wang, J.; Jiang, X. A novel bearing intelligent fault diagnosis framework under time-varying working conditions using recurrent neural network. ISA Trans. 2020, 100, 155–170.
  20. Wang, Y.; Cheng, L. A combination of residual and long–short-term memory networks for bearing fault diagnosis based on time-series model analysis. Meas. Sci. Technol. 2020, 32, 015904.
  21. Li, Z.; Zheng, T.; Wang, Y.; Cao, Z.; Guo, Z.; Fu, H. A novel method for imbalanced fault diagnosis of rotating machinery based on generative adversarial networks. IEEE Trans. Instrum. Meas. 2020, 70, 1–17.
  22. Yang, X.; Liu, B.; Xiang, L.; Hu, A.; Xu, Y. A novel intelligent fault diagnosis method of rolling bearings with small samples. Measurement 2022, 203, 111899.
  23. Wu, Z.; Lin, W.; Fu, B.; Guo, J.; Ji, Y.; Pecht, M. A local adaptive minority selection and oversampling method for class-imbalanced fault diagnostics in industrial systems. IEEE Trans. Reliab. 2019, 69, 1195–1206.
  24. Xiao, D.; Huang, Y.; Qin, C.; Liu, Z.; Li, Y.; Liu, C. Transfer learning with convolutional neural networks for small sample size problem in machinery fault diagnosis. Proc. Inst. Mech. Eng. Part C J. Mech. Eng. Sci. 2019, 233, 5131–5143.
  25. Jiang, C.; Chen, H.; Xu, Q.; Wang, X. Few-shot fault diagnosis of rotating machinery with two-branch prototypical networks. J. Intell. Manuf. 2023, 34, 1667–1681.
  26. Yue, K.; Li, J.; Chen, J.; Huang, R.; Li, W. Multiscale Wavelet Prototypical Network for Cross-Component Few-Shot Intelligent Fault Diagnosis. IEEE Trans. Instrum. Meas. 2022, 72, 1–11.
  27. Qian, Q.; Qin, Y.; Wang, Y.; Liu, F. A new deep transfer learning network based on convolutional auto-encoder for mechanical fault diagnosis. Measurement 2021, 178, 109352.
  28. He, Z.; Shao, H.; Wang, P.; Lin, J.J.; Cheng, J.; Yang, Y. Deep transfer multi-wavelet auto-encoder for intelligent fault diagnosis of gearbox with few target training samples. Knowl. Based Syst. 2020, 191, 105313.
  29. Wang, D.; Zhang, M.; Xu, Y.; Lu, W.; Yang, J.; Zhang, T. Metric-based meta-learning model for few-shot fault diagnosis under multiple limited data conditions. Mech. Syst. Signal Process. 2021, 155, 107510.
  30. Vinyals, O.; Blundell, C.; Lillicrap, T.; Wierstra, D.; Kavukcuoglu, K. Matching networks for one shot learning. Adv. Neural Inf. Process. Syst. 2016, 29.
  31. Wang, R.; Huang, W.; Shi, M.; Wang, J.; Shen, C.; Zhu, Z. Federated adversarial domain generalization network: A novel machinery fault diagnosis method with data privacy. Knowl. Based Syst. 2022, 256, 109880.
  32. Zhao, K.; Hu, J.; Shao, H.; Hu, J. Federated multi-source domain adversarial adaptation framework for machinery fault diagnosis with data privacy. Reliab. Eng. Syst. Saf. 2023, 236, 109246.
  33. Chen, J.; Li, J.; Huang, R.; Yue, K.; Chen, Z.; Li, W. Federated transfer learning for bearing fault diagnosis with discrepancy-based weighted federated averaging. IEEE Trans. Instrum. Meas. 2022, 71, 1–11.
  34. Geng, D.; He, H.; Lan, X.; Liu, C. Bearing fault diagnosis based on improved federated learning algorithm. Computing 2022, 104, 1–19.
  35. Chen, J.; Tang, J.; Chen, J. Federated Meta-Learning Framework for Few-shot Fault Diagnosis in Industrial IoT. In Proceedings of the GLOBECOM 2022 IEEE Global Communications Conference, Rio de Janeiro, Brazil, 4–8 December 2022; IEEE: New York, NY, USA, 2022; pp. 2993–2998.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback