Mental Fatigue Detection Using Physiological Signals: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Jessie Wu and Version 1 by AMINE JAOUADI.

Fatigue is a state characterized by both physical and mental exhaustion, resulting from prolonged activity, inadequate rest, or excessive cognitive demands. Physiological signals offer a valuable insight into the body’s internal state. Monitoring and interpreting these signals provide real-time information about an individual’s physical and mental condition, enabling early fatigue detection. 

  • fatigue detection
  • electrodermal activity
  • feature selection

1. Introduction

Fatigue prevalent phenomenon spans various aspects of life, including professional, academic, and daily routines. Mental fatigue can be defined as reduced cognitive performance due to cognitive overload, resulting from task duration or workload, independent of sleepiness [1]. The consequences of fatigue are significant, including impaired decision-making, an increased risk of accidents, and a general decline in well-being [2,3,4][2][3][4]. Understanding the mechanisms of mental fatigue and developing effective strategies to manage and mitigate its effects are crucial for promoting health, safety, and peak performance in diverse professional settings.
Physiological signals offer a valuable insight into the body’s internal state. Monitoring and interpreting these signals provide real-time information about an individual’s physical and mental condition, enabling early fatigue detection [5]. This capability is particularly relevant in mission-critical environments such as transportation [6,7][6][7], healthcare [8], and industrial settings [9]. Consequently, the study of physiological signals for fatigue assessment is a rapidly advancing field, with far-reaching implications for human performance and well-being.
Despite the numerous approaches proposed for fatigue detection and monitoring, there is no universally accepted gold standard. Current non-invasive methods are primarily based on the following measuring principles: subjective measures, performance-related methods, and physiological signal-based methods.
Subjective Measures: Subjective measures involve self-reported fatigue assessment through questionnaires and scales [10[10][11],11], but they are not suitable for online monitoring. Nevertheless, they offer valuable insights into the mental and emotional processes influencing task performance, serving as valuable benchmarks when comparing results with fatigue models.
Performance-Related Methods: Performance-related methods rely on the fact that an individual’s cognitive and motor performance in specific tasks reflects their fatigue level. These methods consist of conducting tests based on neuro-behavioral tasks to evaluate performance, with a focus on cognitive abilities (e.g., vigilance, reaction time, sustained attention) [12]. Although performance-related methods are easily standardized, they are incapable of real-time fatigue detection for preventing the occurrence of potential incidents.
Physiological Signal-Based Methods: Physiological signal-based methods detect fatigue onset by monitoring subjects’ physiological responses, including brain activity, measured via electroencephalogram (EEG) [13]; heart activity, measured via electrocardiogram (ECG) [14]; and more recently, electrodermal activity (EDA) [15]. Utilizing physiological signals as fatigue indicators allows objective real-time monitoring at the individual level.
These methods are often complemented by machine learning (ML) algorithms to classify outputs as indicative of different fatigue states [16]. These algorithms “learn” meaningful information from physiological signals and/or task performance results to predict corresponding fatigue states. A primary limitation of these algorithms comes from the quality and quantity of the data required for training. In terms of data quality, machine learning models struggle to discern relevant information from noise. Instead, they try to identify the optimal statistical relationships between input data and target outputs. In terms of data quantity, depending on the specific ML algorithm, these models can be more or less data greedy, limiting their applicability in real-world scenarios where obtaining large volumes of physiological data may be challenging.

2. An Overview of the Existing Literature on Mental Fatigue Detection Using Physiological Signals

Numerous studies have demonstrated the relevance of EEG features in mental fatigue detection [17,18,19,20][17][18][19][20]. However, EEGs are often time-consuming and susceptible to environmental electromagnetic interference, making them impractical for real-life environments. Consequently, this led to the exploration of alternative electric extracerebral measurements like ECG and EDA [21]. ECG signals are widely used in estimating mental fatigue, with heart rate variability (HRV) being a key feature for detection [22]. HRV reflects autonomic neural system (ANS) regulation, which alters during stress, fatigue, and drowsiness episodes. HRV is defined as the variation in time intervals between consecutive heartbeats and can be analyzed in both time and frequency domains [23,24,25][23][24][25]. In the time domain, HRV features like the number of beats per minute, mean time interval between heart beats, and standard deviation in beat intervals are widely used. In the frequency domain, the ratio of low-frequency (LF) component (0.04–0.15 Hz) to high-frequency (HF) component (0.15–0.4 Hz) of HRV power spectrum describes the sympathovagal balance, serving as an important marker of cognitive fatigue. Various machine-learning-based fatigue detection approaches can be found in the literature that rely on these ECG features. For instance, ref. [25] implemented a neural-network-based model to detect fatigue using HRV features. Ref. [26] implemented a convolutional neural network (CNN), recurrent neural network, and long short-term memory (RNN-LSTM)-based models for fatigue detection using EEG and ECG signals along with physiognomic data. EDA refers to changes in sweat gland activity that are reflective of the intensity of an individual’s emotional state, due to its close link to the sympathetic nervous system (SNS) [27,28,29][27][28][29]. EDA manifests as continuous changes in skin electrical characteristics. Among the various aspects of EDA, skin conductance (SC) has been one of the most extensively researched. Commonly, the SC signal is deconstructed into two distinct components, namely the tonic and phasic components. The tonic component or skin conductance level (SCL) represents slower-acting aspects of the signal, including background characteristics. SCL variations indicate changes in autonomic arousal, though they can also be influenced by factors unrelated to the sympathetic nervous system, such as temperature fluctuations and physical exercise-induced perspiration. The phasic component or skin conductance response (SCR) overlays SCL and captures rapidly changing aspects of SC. SCR provides moment-to-moment arousal measurement, reflecting responses specific to stimuli or general orienting processes. EDA holds promise for quantifying human cognitive states and has potential real-world applications. Recently, Zeng et al. [30] developed a wearable non-invasive epidermal system for monitoring ECG, EDA, and respiration signals simultaneously. The main advantage of their system is that the device fabrication method is simple and provides a powerful strategy for further development of epidermal multi-functional sensors. In their research, the system’s potential was assessed by conducting a study to detect mental fatigue. This was achieved by utilizing the physiological signal data collected by their system and training machine learning algorithms, like support vector machine (SVM), K-nearest neighbors (KNN), and decision tree (DT), with these data. They used the following physiological signal features: mean heart rate, HRV standard deviation, number of SCR peaks, sum of SCR peak amplitude, sum of SCR peak duration, respiration rate. They achieved a maximum accuracy of 87% using the DT algorithm.

References

  1. Wang, Q.; Yang, J.; Ren, M.; Zheng, Y. Driver Fatigue Detection: A Survey. In Proceedings of the 2006 6th World Congress on Intelligent Control and Automation, Dalian, China, 21–23 June 2006; Volume 2, pp. 8587–8591.
  2. Wang, X.; Xu, C. Driver drowsiness detection based on non-intrusive metrics considering individual specifics. Accid. Anal. Prev. 2016, 95, 350–357.
  3. Brown, I.D. Driver Fatigue. Hum. Factors 1994, 36, 298–314.
  4. Brookhuis, K.A.; de Waard, D. Monitoring drivers’ mental workload in driving simulators using physiological measures. Accid. Anal. Prev. 2010, 42, 898–903.
  5. Dingus, T.A. Development of Models for Detection of Automobile Driver Impairment. Ph.D. Thesis, Faculty of Virginia Polytechnic Institute, Blacksburg, VA, USA, 1985.
  6. Thiffault, P.; Bergeron, J. Monotony of Road Environment and Driver Fatigue: A Simulator Study. Accid. Anal. Prev. 2003, 35, 381–391.
  7. Weng, C.H.; Lai, Y.H.; Lai, S.H. Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network. In Proceedings of the Computer Vision—ACCV 2016 Workshops, Taipei, Taiwan, 20–24 November 2016; Chen, C.S., Lu, J., Ma, K.K., Eds.; Lecture Notes in Computer Science. Springer: Cham, Switzerland, 2017; pp. 117–133.
  8. Benzo, R.M.; Farag, A.; Whitaker, K.M.; Xiao, Q.; Carr, L.J. Examining the Impact of 12-Hour Day and Night Shifts on Nurses’ Fatigue: A Prospective Cohort Study. Int. J. Nurs. Stud. Adv. 2022, 4, 100076.
  9. Givi, Z.; Jaber, M.; Neumann, W. Modelling Worker Reliability with Learning and Fatigue. Appl. Math. Model. 2015, 39, 5186–5199.
  10. Bentall, R.P.; Wood, G.C.; Marrinan, T.; Deans, C.; Edwards, R.H.T. A Brief Mental Fatigue Questionnaire. Br. J. Clin. Psychol. 1993, 32, 375–377.
  11. Shahid, A.; Wilkinson, K.; Marcu, S.; Shapiro, C.M. Visual Analogue Scale to Evaluate Fatigue Severity (VAS-F). In STOP, THAT and One Hundred Other Sleep Scales; Shahid, A., Wilkinson, K., Marcu, S., Shapiro, C.M., Eds.; Springer: New York, NY, USA, 2011; pp. 399–402.
  12. Lee, I.S.; Bardwell, W.A.; Ancoli-Israel, S.; Dimsdale, J.E. Number of Lapses during the Psychomotor Vigilance Task as an Objective Measure of Fatigue. J. Clin. Sleep Med. 2010, 6, 163–168.
  13. Stancin, I.; Cifrek, M.; Jovic, A. A Review of EEG Signal Features and Their Application in Driver Drowsiness Detection Systems. Sensors 2021, 21, 3786.
  14. Hasan, M.M.; Watling, C.N.; Larue, G.S. Physiological signal-based drowsiness detection using machine learning: Singular and hybrid signal approaches. J. Saf. Res. 2022, 80, 215–225.
  15. Heaton, K.J.; Williamson, J.R.; Lammert, A.C.; Finkelstein, K.R.; Haven, C.C.; Sturim, D.; Smalt, C.J.; Quatieri, T.F. Predicting changes in performance due to cognitive fatigue: A multimodal approach based on speech motor coordination and electrodermal activity. Clin. Neuropsychol. 2020, 34, 1190–1214.
  16. Adão Martins, N.R.; Annaheim, S.; Spengler, C.M.; Rossi, R.M. Fatigue Monitoring Through Wearables: A State-of-the-Art Review. Front. Physiol. 2021, 12, 2285.
  17. Yu, S.; Li, P.; Lin, H.; Rohani, E.; Choi, G.; Shao, B.; Wang, Q. Support Vector Machine Based Detection of Drowsiness Using Minimum EEG Features. In Proceedings of the 2013 International Conference on Social Computing, Alexandria, VA, USA, 8–14 September 2013; pp. 827–835.
  18. Khushaba, R.N.; Kodagoda, S.; Lal, S.; Dissanayake, G. Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm. IEEE Trans. Biomed. Eng. 2011, 58, 121–131.
  19. He, Q.; Li, W.; Fan, X.; Fei, Z. Driver fatigue evaluation model with integration of multi-indicators based on dynamic Bayesian network. IET Intell. Transp. Syst. 2015, 9, 547–554.
  20. Guo, Z.; Chen, R.; Zhang, K.; Pan, Y.; Wu, J. The Impairing Effect of Mental Fatigue on Visual Sustained Attention under Monotonous Multi-Object Visual Attention Task in Long Durations: An Event-Related Potential Based Study. PLoS ONE 2016, 11, e0163360.
  21. McNaboe, R.; Beardslee, L.; Kong, Y.; Smith, B.N.; Chen, I.P.; Posada-Quintero, H.F.; Chon, K.H. Design and Validation of a Multimodal Wearable Device for Simultaneous Collection of Electrocardiogram, Electromyogram, and Electrodermal Activity. Sensors 2022, 22, 8851.
  22. Matuz, A.; Van Der Linden, D.; Kisander, Z.; Hernádi, I.; Kázmér, K.; Csathó, Á. Enhanced Cardiac Vagal Tone in Mental Fatigue: Analysis of Heart Rate Variability in Time-on-Task, Recovery, and Reactivity. PLoS ONE 2021, 16, e0238670.
  23. Egelund, N. Spectral analysis of heart rate variability as an indicator of driver fatigue. Ergonomics 1982, 25, 663–672.
  24. Abdul Rahim, H.; Dalimi, A.; Jaafar, H. Detecting Drowsy Driver Using Pulse Sensor. J. Teknol. 2015, 73, 5–8.
  25. Patel, M.; Lal, S.; Kavanagh, D.; Rossiter, P. Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst. Appl. 2011, 38, 7235–7242.
  26. Abbas, Q.; Ibrahim, M.E.; Khan, S.; Baig, A.R. Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System. Comput. Mater. Contin. 2022, 71, 1999–2007.
  27. Wang, D.; Shen, P.; Wang, T.; Xiao, Z. Fatigue Detection of Vehicular Driver through Skin Conductance, Pulse Oximetry and Respiration: A Random Forest Classifier. In Proceedings of the 2017 IEEE 9th International Conference on Communication Software and Networks (ICCSN), Guangzhou, China, 6–8 May 2017; pp. 1162–1166.
  28. Malathi, D.; Dorathi Jayaseeli, J.; Madhuri, S.; Senthilkumar, K. Electrodermal Activity Based Wearable Device for Drowsy Drivers. J. Phys. Conf. Ser. 2018, 1000, 012048.
  29. Williamson, J.R.; Heaton, K.J.; Lammert, A.; Finkelstein, K.; Sturim, D.; Smalt, C.; Ciccarelli, G.; Quatieri, T.F. Audio, Visual, and Electrodermal Arousal Signals as Predictors of Mental Fatigue Following Sustained Cognitive Work. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; pp. 832–836.
  30. Zeng, Z.; Huang, Z.; Leng, K.; Han, W.; Niu, H.; Yu, Y.; Ling, Q.; Liu, J.; Wu, Z.; Zang, J. Nonintrusive Monitoring of Mental Fatigue Status Using Epidermal Electronic Systems and Machine-Learning Algorithms. ACS Sens. 2020, 5, 1305–1313.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
ScholarVision Creations
Feedback