Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1
Ling Wang
 -- 1425 2023-11-06 17:33:27 |
2 format correct
Catherine Yang
 Meta information modification 1425 2023-11-14 03:12:43 |

Video Upload Options

We provide professional Video Production Services to translate complex research into visually appealing presentations. Would you like to try it?
No, upload directly Yes

Confirm

Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Wang, L.; Hao, J.; Zhou, T.H. Electrocardiogram-Based Emotion Recognition. Encyclopedia. Available online: https://encyclopedia.pub/entry/51206 (accessed on 19 November 2024).
Wang L, Hao J, Zhou TH. Electrocardiogram-Based Emotion Recognition. Encyclopedia. Available at: https://encyclopedia.pub/entry/51206. Accessed November 19, 2024.
Wang, Ling, Jiayu Hao, Tie Hua Zhou. "Electrocardiogram-Based Emotion Recognition" Encyclopedia, https://encyclopedia.pub/entry/51206 (accessed November 19, 2024).
Wang, L., Hao, J., & Zhou, T.H. (2023, November 06). Electrocardiogram-Based Emotion Recognition. In Encyclopedia. https://encyclopedia.pub/entry/51206
Wang, Ling, et al. "Electrocardiogram-Based Emotion Recognition." Encyclopedia. Web. 06 November, 2023.
Electrocardiogram-Based Emotion Recognition
Edit
This entry is adapted from the peer-reviewed paper 10.3390/s23208636

Heart rate variability (HRV) serves as a significant physiological measure that mirrors the regulatory capacity of the cardiac autonomic nervous system. It not only indicates the extent of the autonomic nervous system’s influence on heart function but also unveils the connection between emotions and psychological disorders. 

heart rate variability emotion recognition

1. Introduction

Emotions are innate physiological responses designed to maintain or restore homeostasis by altering the environment for more appropriate interactions [1]. Emotions are not only significant in the realm of social understanding and interaction, but they also trigger physiological mechanisms in order to proficiently recognize and address a diverse range of challenges and opportunities, thus contributing to our survival and growth [2]. Emotional experiences are closely linked to bodily perceptions, and they help direct attention to critical events, such as physical needs, potential threats, and social interactions. Emotional experiences play a role in coordinating behavioral and physiological responses during these critical occasions [3].
The autonomic nervous system (ANS) plays a significant role in conveying extensive data on emotional states, is tightly connected to emotional arousal, and at the same time, dominates the function of the lungs, heart, and numerous other organ systems [4]. Various mood-related physiological variables are often easily measured and thus detect altered states of the ANS [5]. Such changes usually occur unknowingly. For some stimuli and unconscious responses, the state of the response changes more rapidly than the conscious response. These physical aspects include facial expression, movement and gait, voice characteristics, electrocardiogram, electroencephalogram, electrooculogram, and galvanic skin response [6][7][8][9]. Of particular interest is the fact that the core area of our focus, the electrocardiogram (ECG), has been shown to have a strong correlation with emotional characteristics as well as with ECG waveforms. Numerous scholars have explored the feasibility and limitations of utilizing ECG signals for emotion detection [10]. Furthermore, recent developments in machine learning and deep learning have shown that emotion recognition systems can reliably derive information from ECG data [11]. Most sentiment recognition tasks involving machine learning or deep learning predominantly employ a fully supervised learning paradigm [12]. There are several limitations to this approach. First, in a typical fully supervised learning scenario, each classification or regression task requires re-training the model from scratch, which requires significant computational resources and time. In addition, features obtained from a fully supervised trained model are usually too task-specific to generalize well to other tasks. Finally, typical fully supervised learning usually requires training with large-scale manually labeled datasets, as small datasets usually lead to the performance degradation of deep networks.
The autonomic nervous system (ANS) consists of the sympathetic and parasympathetic branches, working in harmony to oversee the functioning of target organs and tissues, ensuring the preservation of homeostasis [13]. Emotion is closely connected to these branches as they all provide nerve signals to the heart, with a particular emphasis on the AV node. This node collaborates with neuromodulation to facilitate the coordination of neurotransmitters, ensuring they work in harmony to regulate their functions and generate a distinct response to emotional conditions. Sympathetic overactivity may result in additional cardiac contractions or a rapid heartbeat. In contrast, parasympathetic responses to negative emotions are often influenced by olfactory or visual stimuli, which may result in a slowed heartbeat or even cardiac arrest [14]. The spectral examination of heart rate variability (HRV) is considered a non-intrusive technique to assess the balance between the two principal branches of the autonomic nervous system (ANS), which has been suggested for detecting and characterizing emotional states [15]. A set of guidelines has been formulated for evaluating, physiologically interpreting, and clinically applying resting heart rate variability (HRV), which includes the delineation of three distinct spectral components: an ultra-low-frequency (ULF) spectrum ranging from 0 to 0.04 Hz, a low-frequency (LF) spectrum covering the range from 0.04 to 0.15 Hz, and a high-frequency (HF) spectrum spanning from 0.15 to 0.4 Hz. HF band power is regarded as an indicator of parasympathetic activity, primarily attributed to respiratory sinus arrhythmia. The power within the low-frequency range is regarded as an indicator of both the sympathetic and parasympathetic activity of the heart. HRV serves as a mirror to the fluctuations in the autonomic nervous system, which, in turn, can mirror the accompanying emotional states.

2. Electrocardiogram-Based Emotion Recognition 

Presently, the primary focus in emotion recognition through ECG revolves around the utilization of classification algorithms alongside the selection of pertinent features and the scope of identifiable emotions. Machine learning algorithms have proven to be effective for emotion recognition based on ECG data. Axel and his team employed the wavelet scattering method for the extraction of unique characteristics from ECG data. This method facilitated the acquisition of signal features spanning varying time scales to assess their performance. The outcomes of this study demonstrated that the proposed feature-extraction and signal classification algorithm, within the realm of emotional dimensions, achieved an accuracy rate of 88.8% for arousal, 90.2% for valence, and an impressive 95.3% for two-dimensional classification [16]. Theekshana et al. presented a machine learning approach utilizing an ensemble learning method to identify core emotions, including anger, sadness, happiness, and the combination of happiness with electrocardiogram (ECG) data. They utilized spectral analysis as a novel approach to extract features and assessed the efficacy of a widely recognized collection of ensemble learners for emotion classification through a machine learning process. The ensemble learners exhibited a 10.77% enhancement in accuracy when compared to the top-performing individual biosensor-based model [17]. Yan et al. introduced the X-GWO-SVM approach to analyze sentiments from ECG data. They conducted single-subject cross-validation and achieved an impressive average accuracy rate of 95.93% when utilizing the WESAD dataset. This result demonstrates its superior reliability compared to previous implementations of supervised machine learning techniques [18].
Deep learning algorithms are commonly employed in the realm of emotion recognition as well. Pritam et al. devised a novel approach using a self-supervised deep multitask learning framework to detect emotions based on electrocardiogram (ECG) data. In this process, the convolutional layer remained fixed, while the dense layer was fine-tuned using labeled ECG data. Notably, this innovative solution yielded state-of-the-art results in classifying arousal, mood, emotional state, and stress across four distinct datasets [19]. Xu et al. introduced an emotion detection technique rooted in deep learning tailored for healthcare data analysis. Their approach incorporated a multichannel convolutional neural network to extract distinctive features from ECG data and textual content related to emotions, particularly for detecting emotional fatigue. Ultimately, this method amalgamated the features derived from multiple data sources to ascertain emotional states. The experimental findings demonstrated that the proposed model consistently achieved an accuracy rate exceeding 85% in the prediction of emotional fatigue [20]. Chen et al. introduced a novel approach to emotion recognition, which involved combining multiple sensory modalities using the Dempster–Shafer evidence theory. They utilized an SVM classifier to categorize EEG signal features. The results of the experiment demonstrated that the multimodal fusion model outperformed the unimodal emotion detection method, resulting in a significant increase in accuracy by 7.37% and 8.73% as compared to the emotion recognition model based on ECG signals [21]. Hammad et al. applied the PETSFCNN model in their research on emotion recognition, achieving an impressive maximum classification accuracy of 97.56%. They employed a deep neural network in combination with grid search optimization to enhance accuracy in classification tasks. The results of their study indicated that the suggested method surpasses existing techniques in precisely identifying emotions from ECG signals. This implies the possibility of utilizing it as an intelligent system for emotion recognition [22].
When it comes to selecting features for emotion recognition, HRV-related characteristics have gained widespread usage. Guo et al. conducted an analysis of ECG signals to extract heart rate variability (HRV) features, employing techniques encompassing frequency domain, time domain, statistical methods, and Poincaré. The HRV characteristics were later utilized for categorizing different emotional states, utilizing principal component analysis, and then employing a support vector machine to reduce the feature set. Notably, they achieved classification accuracies of 71.4% for distinguishing two emotional states (positive/negative) and 56.9% for classifying five emotional states [23]. Ferdinando et al. employed K-nearest neighbors (KNN) as a classifier in their study. They combined supervised dimensionality reduction using neighborhood component analysis (NCA) with feature-extraction techniques, which included capturing standard HRV features and statistical distributions of instantaneous frequencies. These methods were applied to address the classification of three distinct emotion and arousal categories. The findings suggested that, in most instances, integrating NCA resulted in a significant performance boost of 74% when compared to the absence of NCA in the implementation [24]. Singson et al. harnessed a ResNet-based CNN to analyze both facial expressions and physiological data, with a particular focus on heart rate variability (HRV) features. Their aim was to discern and validate emotions, achieving an accuracy rate of 68.42% through the analysis of ECG signals [25].

References

  1. Cai, Y.; Li, X.; Li, J. Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review. Sensors 2023, 23, 2455.
  2. Ford, B.Q.; Lam, P.; John, O.P.; Mauss, I.B. The psychological health benefits of accepting negative emotions and thoughts: Laboratory, diary, and longitudinal evidence. J. Personal. Soc. Psychol. 2018, 115, 1075.
  3. Jerath, R.; Beveridge, C. Respiratory rhythm, autonomic modulation, and the spectrum of emotions: The future of emotion recognition and modulation. Front. Psychol. 2020, 11, 1980.
  4. Candia-Rivera, D.; Catrambone, V.; Thayer, J.F.; Gentili, C.; Valenza, G. Cardiac sympathetic-vagal activity initiates a functional brain–body response to emotional arousal. Proc. Natl. Acad. Sci. USA 2022, 119, e2119599119.
  5. Mulkey, S.B.; du Plessis, A.J. Autonomic nervous system development and its impact on neuropsychiatric outcome. Pediatr. Res. 2019, 85, 120–126.
  6. Wijasena, H.Z.; Ferdiana, R.; Wibirama, S. A survey of emotion recognition using physiological signal in wearable devices. In Proceedings of the 2021 International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), Delft, The Netherlands, 12–16 July 2021; pp. 1–6.
  7. Raheel, A.; Majid, M.; Alnowami, M.; Anwar, S.M. Physiological sensors based emotion recognition while experiencing tactile enhanced multimedia. Sensors 2020, 20, 4037.
  8. Quintana, D.S.; Guastella, A.J.; Outhred, T.; Hickie, I.B.; Kemp, A.H. Heart rate variability is associated with emotion recognition: Direct evidence for a relationship between the autonomic nervous system and social cognition. Int. J. Psychophysiol. 2012, 86, 168–172.
  9. Jemioło, P.; Storman, D.; Mamica, M.; Szymkowski, M.; Żabicka, W.; Wojtaszek-Główka, M.; Ligęza, A. Datasets for Automated Affect and Emotion Recognition from Cardiovascular Signals Using Artificial Intelligence—A Systematic Review. Sensors 2022, 22, 2538.
  10. Hasnul, M.A.; Aziz, N.A.A.; Alelyani, S.; Mohana, M.; Aziz, A.A. Electrocardiogram-based emotion recognition systems and their applications in healthcare—A review. Sensors 2021, 21, 5015.
  11. Saganowski, S. Bringing emotion recognition out of the lab into real life: Recent advances in sensors and machine learning. Electronics 2022, 11, 496.
  12. Ferdinando, H.; Seppänen, T.; Alasaarela, E. Enhancing Emotion Recognition from ECG Signals using Supervised Dimensionality Reduction. In Proceedings of the Pattern Recognition Applications and Methods: 6th International Conference, ICPRAM 2017, Porto, Portugal, 24–26 February 2017; pp. 112–118.
  13. Rowe, J.A.; McMurphy, R.M.; Lutjemeier, B.J.; Kenney, M.J. Introduction to the Autonomic Nervous System and Autonomic Pharmacology. In Veterinary Pharmacology and Therapeutics; Wiley Blackwell: Hoboken, NJ, USA, 2018; pp. 113–130.
  14. Valderas, M.T.; Bolea, J.; Laguna, P.; Vallverdú, M.; Bailón, R. Human emotion recognition using heart rate variability analysis with spectral bands based on respiration. In Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 25–29 August 2015; pp. 6134–6137.
  15. Zhu, J.; Ji, L.; Liu, C. Heart rate variability monitoring for emotion and disorders of emotion. Physiol. Meas. 2019, 40, 064004.
  16. Sepúlveda, A.; Castillo, F.; Palma, C.; Rodriguez-Fernandez, M. Emotion recognition from ECG signals using wavelet scattering and machine learning. Appl. Sci. 2021, 11, 4945.
  17. Dissanayake, T.; Rajapaksha, Y.; Ragel, R.; Nawinne, I. An ensemble learning approach for electrocardiogram sensor based human emotion recognition. Sensors 2019, 19, 4495.
  18. Yan, X.; Lin, Z.; Lin, Z.; Vucetic, B. A Novel Exploitative and Explorative GWO-SVM Algorithm for Smart Emotion Recognition. IEEE Internet Things J. 2023, 10, 9999–10011.
  19. Sarkar, P.; Etemad, A. Self-supervised ECG representation learning for emotion recognition. IEEE Trans. Affect. Comput. 2020, 13, 1541–1554.
  20. Xu, J.; Hu, Z.; Zou, J.; Bi, A. Intelligent emotion detection method based on deep learning in medical and health data. IEEE Access 2019, 8, 3802–3811.
  21. Chen, T.; Yin, H.; Yuan, X.; Gu, Y.; Ren, F.; Sun, X. Emotion recognition based on fusion of long short-term memory networks and SVMs. Digit. Signal Prog. 2021, 117, 103153.
  22. Hammad, D.S.; Monkaresi, H. ECG-Based Emotion Detection via Parallel-Extraction of Temporal and Spatial Features Using Convolutional Neural Network. Trait. Signal 2022, 39, 43.
  23. Guo, H.-W.; Huang, Y.-S.; Lin, C.-H.; Chien, J.-C.; Haraikawa, K.; Shieh, J.-S. Heart rate variability signal features for emotion recognition by using principal component analysis and support vectors machine. In Proceedings of the 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE), Taichung, Taiwan, 31 October–2 November 2016; pp. 274–277.
  24. Ferdinando, H.; Seppänen, T.; Alasaarela, E. Emotion recognition using neighborhood components analysis and ecg/hrv-based features. In Proceedings of the Pattern Recognition Applications and Methods: 6th International Conference, ICPRAM 2017, Porto, Portugal, 24–26 February 2017; Revised Selected Papers 6. Springer International Publishing: Cham, Switzerland, 2018; pp. 99–113.
  25. Singson, L.N.B.; Sanchez, M.T.U.R.; Villaverde, J.F. Emotion recognition using short-term analysis of heart rate variability and resnet architecture. In Proceedings of the 2021 13th International Conference on Computer and Automation Engineering (ICCAE), Melbourne, Australia, 20–22 March 2021; pp. 15–18.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Computer Science, Artificial Intelligence
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Ling Wang
,
Jiayu Hao
,
Tie Hua Zhou
View Times: 340
Revisions: 2 times (View History)
Update Date: 14 Nov 2023
Table of Contents
    1000/1000
    Hot Most Recent
    ScholarVision Creations
    Feedback