Table of Contents

    Topic review

    Sleep Stages Detection Using DL

    Subjects: Others
    View times: 36
    Contributors: Oliver Faust , Chui Ping Ooi
    Submitted by: Oliver Faust


    Sleep is vital for one’s general well-being, but is often neglected, which has led to an increase in sleep disorders worldwide. Indicators of sleep disorders, such as sleep interruptions, extreme daytime drowsiness, or snoring, can be detected with sleep analysis. However, sleep analysis relies on visuals conducted by experts, and is susceptible to inter- and intra-observer variabilities. One way to overcome these limitations is to support experts with a programmed diagnostic tool (PDT) based on artificial intelligence for timely detection of sleep disturbances. Artificial intelligence technology, such as deep learning (DL), ensures that data are fully utilized with low to no information loss during training.

    1. Introduction

    Sleep is crucial for the maintenance and regulation of various biological functions at a molecular level [1], which helps humans to restore physical and mental wellbeing and proper brain function during the day [2]. There are two primary types of sleep: non-rapid eye movement (NREM) and rapid eye movement (REM) sleep. NREM sleep comprises four stages, after which, it continues into the REM sleep stage. NREM and REM sleep stages are connected and cyclically alternated through the sleep process wherein unbalanced cycling or the absence of sleep stages give rise to sleep disorders [3]. Unfortunately, sleep disorders, which lead to poor sleep quality, are often neglected [4]. Stranges et al. [4] highlighted that sleep-related problems is a looming global health issue. In their study, datasets from the World Health Organization (WHO) and International Network for the Demographic Evaluation of Populations and Their Health (INDEPTH) were used to investigate the prevalence of sleep problems in low-income countries. It was reported that 16.6% of the adult population, which amounts to approximately 150 million, have sleep problems and current trends indicate that this figure will increase to 260 million by 2030.

    To date, it is mandatory that sleep stage scoring is done manually by human experts [5][6]. However, human experts have limited capacity to handle slow changes in background electroencephalography (EEG) and learn the different rules to score sleep stages for various polysomnogram (PSG) recordings [6]. Furthermore, evaluations by human experts are prone to inter- and intra-observer variabilities that can negatively affect the quality of sleep stage scoring [7]. Other important factors affecting sleep stage scoring are patient convenience and diagnosis cost. As such, a sleep lab is a highly controlled environment that requires dedicated facilities and highly trained personnel. Hence, sleep labs tend to be in urban centers and patients must travel there to spend one or multiple nights in the facility. These factors make sleep labs inconvenient for patients and the cost per diagnosis is high. Other diagnostic methods, such as portable monitoring devices for sleep stages, exhibit some advantages, such as enhancing access to patients, low cost, and user-friendliness. However, these advantages are outweighed by several disadvantages, such as having diagnostic limitations, failure of device, reliability concerns, and underestimating the apnea/hypopnea index, amongst others [8]. To improve the situation requires a fundamental change in the sleep stage scoring process. We need machines to replace the labor carried out by human experts. This can only be done with systems that understand sleep stages in much of the same way as human experts do. Deep learning (DL) is hailed as a method to mechanize knowledge work, such as sleep stage scoring. However, before we join and adopt this technology, it is prudent to investigate both capabilities and limitations of current DL methods.

    2. Sleep Stages Classification Using DL Models

    2.1. Different Stages of Sleep

    According to Rechtschaffen and Kales (R and K) [9], humans can experience six discrete stages during sleep: (1) wakefulness (W), (2) rapid eye movement (REM) sleep, and (3) four stages of non-REM (NREM) sleep (S1 to S4) [10]. Based on the sleep electroencephalogram (EEG) characteristics, W occurs when the brain is most active, which is represented by high frequency of alpha rhythms. In the NREM sleep, these alpha rhythms eventually diminish when entering the S1 wherein theta rhythm dominates instead. In the S2, sleep spindles and occasional K-complex waveform will appear. The K-complex waveform usually lasts for approximately 1 to 2 s. The S3 sleep occurs when low frequency delta rhythms appear intermittently and eventually, they dominate in the S4 sleep. Finally, REM sleep usually follows after the S4 sleep. In the REM sleep, theta rhythms resurface again, but unlike in the S1 sleep, theta rhythms are accompanied with EEG flattening [10]. Following the guidelines from American Academy of Sleep Medicine (AASM), the S3 and S4 sleep stages can be merged into one sleep stage S3, because of the similarity in their characteristics [11]. Since the delta rhythms are the slowest EEG waves, S3 and S4 sleep stages are known as Slow Wave Sleep (SWS) or the deep sleep. Thus, most sleep classification studies are based on five: W, S1, S2, S3, and REM sleep stages, instead of six (Figure 1).

    Figure 1. Examples of electroencephalography (EEG) signals in different sleep stages.

    2.2. Sleep Databases

    Eight main sleep databases have been used for automated sleep stage classifications. Five of the databases are free to download from PhysioNet [12], namely the Sleep-EDF [13], the expanded Sleep-EDF [13], the St. Vincent’s University Hospital/University College Dublin Sleep Apnea Database (UCD) [12], the Sleep Heart Health Study (SHHS) [14][15], and the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) [16] database. The ISRUC-Sleep datasets [17] can be downloaded from the official websites. Permission is required to obtain the sleep datasets from the Montreal Archive of Sleep Studies (MASS) [18].

    The PSG recordings, in most of the sleep databases, are scored according to R and K rules [9], wherein scoring is done based on wakefulness, NREM sleep and REM sleep. NREM sleep is then subdivided into four stages (S1 to S4). Exceptions are ISRUC and MASS which follow the AASM guideline and partition the recordings into five sleep stages instead of six [11].

    2.3. DL Techniques Used in Automatic Sleep Stage Classification

    The development of a program diagnostic tool (PDT) for automatic sleep stage classification using DL techniques is shown in Figure 2. First, PSG recordings have to be pre-processed to achieve standardization or normalization. Depending on the requirement and architecture of the proposed DL model, additional steps to convert the PSG recordings into the right input format is required; for example, converting one-dimensional (1D) signals into a two-dimensional (2D) format to train 2D-CNN models. Subsequently, the pre-processed signals are split into training, validation, and testing sets. The training set is used to train the model, the validation set is to fine-tune the model, and the testing set is used to evaluate the model’s performance. A well-trained model can accurately classify PSG recordings into the five sleep stages.

    Figure 2. Programmed diagnostic tool (PDT) block diagram with DL for automated sleep stage classification.

    Figure 3 illustrates the number of times each sleep database had been used by studies for automated sleep stage classification using DL techniques, from 2010 to 2020. The DL methods and accuracy obtained from the respective sleep databases are summarized as follows: Sleep-EDF (Table 1), expanded Sleep-EDF (Table 2), MASS (Table 3), MIT-BIH, and SHHS (Table 4), and studies that used the remaining two sleep databases (ISRUC and UCD) and private datasets are listed in Table 5. With the exception of three studies [19][20][21], which classified sleep into four stages, all automated sleep stage classification studies, in Table 1Table 2Table 3Table 4 and Table 5, followed the AASM guidelines [11] and classified sleep into five stages. In studies with sleep databases following the R and K rules [9], (i.e., Sleep-EDF, expanded Sleep-EDF, UCD, SHHS, and MIT-BIH), the S3 and S4 stages were often combined manually before pre-processing the PSG signals.

    Table 1. Summary of automated sleep stage classification approaches with DL applied to PSG recordings in Sleep-EDF dataset.

    Author Signals Samples Approach Tools/Programming Languages Accuracy (%)
    Zhu et al. [22] 2020 EEG 15,188 attention CNN 93.7
    Qureshi et al. [23] 2019 EEG 41,900 CNN 92.5
    Yildirim et al. [24] 2019 EEG 15,188 1D-CNN Keras 90.8
    Hsu et al. [25] 2013 EEG 2880 Elman RNN 87.2
    Michielli et al. [26] 2019 EEG 10,280 RNN-LSTM MATLAB 86.7
    Wei et al. [27] 2017 EEG CNN 84.5
    Mousavi et al. [28] 2019 EEG 42,308 CNN-BiRNN TensorFlow 84.3
    Seo et al. [29] 2020 EEG 42,308 CRNN PyTorch 83.9
    Zhang et al. [30] 2020 EEG CNN 83.6
    Supratak et al. [31] 2017 EEG 41,950 CNN-BiLSTM TensorFlow 82.0
    Phan et al. [32] 2019 EEG Multi-task CNN TensorFlow 81.9
    Vilamala et al. [33] 2017 EEG CNN 81.3
    Phan et al. [34] 2018 EEG 1-max CNN 79.8
    Phan et al. [34] 2018 EEG Attentional RNN 79.1
    Yildirim et al. [24] 2019 EOG 15,188 1D-CNN Keras 89.8
    Yildirim et al. [24] 2019 EEG + EOG 15,188 1D-CNN Keras 91.2
    Xu et al. [35] 2020 PSG signals DNN 86.1
    Phan et al. [32] 2019 EEG + EOG Multi-task CNN TensorFlow 82.3

    Figure 3. Pie chart representation of the frequency in which each sleep database was used in automated sleep stage classification studies. The total number of studies was 47, as listed in Table 1Table 2Table 3Table 4 and Table 5. * Summary statistics: using various databases for sleep stage classification.

    Table 2. Summary of automated sleep stage classification approaches with DL applied to PSG recordings in Expanded Sleep-EDF dataset.

    Author Signals Samples Approach Tools/Programming Languages Accuracy (%)
    Wang et al. [36] 2018 EEG C-CNN
    Wang et al. [36] 2018 EEG RNN-biLSTM
    Fernandez-Blanco et al. [37] 2020 EEG CNN 92.7
    Yildirim et al. [24] 2019 EEG 127,512 1D-CNN Keras 90.5
    Jadhav et al. [38] 2020 EEG 62,177 CNN 83.3
    Zhu et al. [22] 2020 EEG 42,269 attention CNN 82.8
    Mousavi et al. [28] 2019 EEG 222,479 1D-CNN TensorFlow 80.0
    Tsinalis et al. [39] 2016 EEG 2D-CNN Lasagne + Theano 74.0
    Yildirim et al. [24] 2019 EOG 127,512 1D-CNN Keras 88.8
    Yildirim et al. [24] 2019 EEG + EOG 127,512 1D-CNN Keras 91.0
    Sokolovsky et al. [40] 2019 EEG + EOG CNN TensorFlow + Keras 81.0

    Table 3. Summary of automated sleep stages classification approaches with DL applied to PSG recordings in Montreal Archive of Sleep Studies (MASS) dataset.

    Author Signals Samples Approach Tools/Programming Languages Accuracy (%)
    Seo et al. [29] 2020 EEG 57,395 CRNN PyTorch 86.5
    Supratak et al. [31] 2017 EEG 58,600 CNN-BiLSTM TensorFlow 86.2
    Phan et al. [32] 2019 EEG Multi-task CNN TensorFlow 78.6
    Dong et al. [41] 2018 EOG F4 MNN RNN-LSTM Theano 85.9
    Dong et al. [41] 2018 EOG Fp2 MNN RNN-LSTM Theano 83.4
    Chambon et al. [42] 2018 EEG/EOG + EMG 2D-CNN Keras
    Phan et al. [43] 2019 EEG + EOG + EMG Hierarchical RNN TensorFlow 87.1
    Phan et al. [32] 2019 EEG + EOG + EMG Multi-task CNN TensorFlow 83.6
    Phan et al. [32] 2019 EEG + EOG Multi-task CNN TensorFlow 82.5

    Table 4. Summary of automated sleep stage classification approaches with DL applied to PSG recordings in Sleep Heart Health Study (SHHS) and Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) datasets.

    Database Author Signals Samples Approach Tools/Programming Languages Accuracy (%)
    MIT-BIH Zhang et al. [44] 2020 EEG Orthogonal CNN 87.6
    Zhang et al. [45] 2018 EEG CUCNN MATLAB 87.2
    SHHS Sors et al. [46] 2018 EEG 5793 CNN 87.0
    Seo et al. [29] 2020 EEG 5,421,338 CRNN PyTorch 86.7
    Fernández-Varela et al. [47] 2019 EEG + EOG + EMG 1,209,971 1D-CNN 78.0
    Zhang et al. [48] 2019 EEG + EOG + EMG 5793 CNN-LSTM
    SHHS Li et al. [19] 2018 ECG HRV 400,547 CNN MATLAB 65.9
    MIT-BIH Li et al. [19] 2018 ECG HRV 2829 CNN MATLAB 75.4
    Tripathy et al. [20] 2018 EEG + HRV 7500 DNN Autoencoder MATLAB 73.7

    Table 5. Summary of automated sleep stage classification approaches with DL applied to PSG recordings in ISRUC, Massachusetts General Hospital (MGH), and University College Dublin Sleep Apnea Database (UCD) datasets.

    Database Author Signals Samples Approach Tools/Programming Languages Accuracy (%)
    ISRUC Cui et al. [49] 2018 EEG CNN 92.2
    Yang et al. [50] 2018 EEG CNN-LSTM
    UCD Zhang et al. [44] 2020 EEG Orthogonal CNN 88.4
    Zhang et al. [45] 2018 EEG CUCNN MATLAB 87.0
    Yuan et al. [51] 2019 Multivariate PSG signals 287,840 Hybrid CNN PyTorch 74.2
    Private datasets Zhang et al. [30] 2020 EEG 264,736 CNN 96.0
    Biswal et al. [52] 2018 PSG signals 10,000 RCNN PyTorch 87.5
    Biswal et al. [53] 2017 EEG 10,000 RCNN TensorFlow 85.7
              Class = 4
    Radha et al. [21] 2019 ECG HRV 541,214 LSTM 77.0

    * The accuracy scores in Table 1Table 2Table 3Table 4 and Table 5 are based on AASM guidelines, five class classification [21].

    Figure 4 shows the number of times PSG recordings such as EEG, EOG, EMG, and ECG signals were used for sleep stage classification studies. It is not surprising that EEG signal was the most popular input for DL models. The characteristic waves and description of each sleep stages are often based on EEG characteristics (i.e., alpha waves, theta waves, delta waves, etc.); Figure 1.

    Figure 4. Different subsets of PSG recordings used to train DL models for automated sleep stage classification as listed in Table 1Table 2Table 3Table 4 and Table 5. Of the 36 studies, the mixture of signals (electrooculogram (EOG), electromyogram (EMG), and electroencephalography (EEG)) was employed 14 times while EEG signals were used 28 times. Only a small fraction (five studies) employed ECG or EOG time series. * Summary statistics: using EEG versus EEG + additional signals.

    Nonetheless, other signals within the PSG recordings are indispensable, because they provide additional information on biological aspects of sleep that may not be manifested in EEG recordings. Since REM sleep is characterized by the movement of eyes and loss in muscle tone of the body core, EOG, and EMG signals may provide key information to separate the REM sleep stage from the other stages. It was shown that some of the REM sleep stages could be overlooked in single-channel EEG input [54]. Therefore, a combination of signals, comprising of EOG, EMG, and EEG, are second in terms of frequency of use after single-channel EEG inputs (Figure 4).

    Although ECG is an important sleep parameter [55], it is not common to use raw ECG signals as a direct input for DL models. As seen in Table 4, heart rate variability (HRV) parameters derived from ECG signals, were used to train the DL models instead. There are only three studies that employed HRV parameters, and these studies classified sleep into four stages instead of five: wakefulness (W), light sleep (S1 and S2), deep sleep (S3 and S4), and REM sleep. Li et al. [19] proposed a 3-layer CNN model. They used a cardiorespiratory coupling (CRC) spectrogram, which was derived from ECG and HRV. Besides alternations in physiological signals, there are other changes in body system changes in some individuals such as cardiovascular [56], respiratory [57], or blood flow in the brain [58]. Hence, the CRC picks up the cardiovascular and respiratory changes. Their model achieved an overall accuracy of 65.9% and 75.4% for SHHS and MIT-BIH respectively, as seen in Table 4. Tripathy et al. [20] combined EEG and HRV features as input to an AE model. During testing, the model achieved an overall accuracy of 73.7%. Radha et al. [21] published the only study that was based on ECG signals from a private dataset that was collected as part of the European Union SIESTA project [59] as shown in Table 5. Likewise, they converted ECG signals into HRV and used the HRV features to train an LSTM model, which achieved an accuracy of 77.0%.

    3. Conclusions

    Sleep disorders are a pressing global issue and the most dangerous sleep disorder is obstructive sleep apnea, which can lead to cardiovascular diseases, if left untreated. Hence, efficient, and accurate diagnostic tools are required for early interventions. In this work, we reviewed 36 studies that employed programmed diagnostic tools with the DL models as the backbone, analyzing overnight polysomnogram recordings to classify sleep stages. Presently, CNN models can offer higher performance in classifying sleep stages, especially with EEG signals. Hence, they are consistently and favourably used by researchers to classify sleep stages as compared to the other machine learning models and physiological signals. Moreover, employing 1D-CNN models is advantageous, because they yield high classification results on EEG signals. However, EEG signals alone may not be sufficient to achieve robust classifications. To achieve robustness and high accuracy one could develop a system that takes advantage of both automated processing and human expert analysis for the interpretation of EEG, EOG, and EMG signals when classifying sleep stages. Therefore, in this review, we highlighted that future studies should focus on classifying sleep stages using all or a combination of these signals. Furthermore, other DL models, such as RNN/LSTM and hybrid models, should also be explored as their full potential has yet to be realized. Future studies could focus on the compatibility and applicability of the DL models in mobile and real time applications. Lastly, more research in developing DL models to detect sleep microstructures is required, as these are often undetected in sleep stage scoring.

    The entry is from 10.3390/app10248963


    1. Laposky, A.; Bass, J.; Kohsaka, A.; Turek, F.W. Sleep and circadian rhythms: Key components in the regulation of energy metabolism. FEBS Lett. 2007, 582, 142–151.
    2. Cho, J.W.; Duffy, J.F. Sleep, sleep disorders, and sexual dysfunction. World J. Men’s Health 2019, 37, 261–275.
    3. Institute of Medicine (US), Committee on Sleep Medicine and Research. Sleep Disorders and Sleep Deprivation: An Unmet Public Health Problem; Colten, H.R., Altevogt, B.M., Eds.; National Academies Press: Washington, DC, USA, 2006.
    4. Stranges, S.; Tigbe, W.; Gómez-Olivé, F.X.; Thorogood, M.; Kandala, N.-B. Sleep problems: An emerging global epidemic? Findings from the INDEPTH WHO-SAGE study among more than 40,000 older adults from 8 countries across Africa and Asia. Sleep 2012, 35, 1173–1181.
    5. Schulz, H. Rethinking sleep analysis. J. Clin. Sleep Med. 2008, 4, 99–103.
    6. Spriggs, W.H. Essentials of Polysomnography; Jones & Bartlett Learning: Burlington, MA, USA, 2014.
    7. Silber, M.H.; Ancoli-Israel, S.; Bonnet, M.H.; Chokroverty, S.; Grigg-Damberger, M.M.; Hirshkowitz, M.; Kapen, S.; A Keenan, S.; Kryger, M.H.; Penzel, T.; et al. The visual scoring of sleep in adults. J. Clin. Sleep Med. 2007, 3, 121–131.
    8. Corral, J.; Pepin, J.-L.; Barbé, F. Ambulatory monitoring in the diagnosis and management of obstructive sleep apnoea syndrome. Eur. Respir. Rev. 2013, 22, 312–324.
    9. Hori, T.; Sugita, Y.; Koga, E.; Shirakawa, S.; Inoue, K.; Uchida, S.; Kuwahara, H.; Kousaka, M.; Kobayashi, T.; Tsuji, Y.; et al. Proposed sments and amendments to ‘A Manual of Standardized Terminology, Techniques and Scoring System for Sleep Stages of Human Subjects’, the Rechtschaffen & Kales (1968) standard. Psychiatry Clin. Neurosci. 2001, 55, 305–310.
    10. Carley, D.W.; Farabi, S.S. Physiology of sleep. Diabetes Spectr. 2016, 29, 5–9.
    11. Iber, C.; Ancoli-Israel, S.; Chesson, A.L.; Quan, S.F. The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specification; American Academy of Sleep Medicine: Darien, IL, USA, 2007.
    12. Goldberger, A.L.; Amaral, L.A.N.; Glass, L.; Hausdorff, J.M.; Ivanov, P.C.; Mark, R.G.; Mietus, J.E.; Moody, G.B.; Peng, C.-K.; Stanley, H.E. PhysioBank, PhysioToolkit, and PhysioNet. Circulation 2000, 101, e215–e220.
    13. Kemp, B.; Zwinderman, A.; Tuk, B.; Kamphuisen, H.; Oberye, J. Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the EEG. IEEE Trans. Biomed. Eng. 2000, 47, 1185–1194.
    14. Zhang, G.-Q.; Cui, L.; Mueller, R.; Tao, S.; Kim, M.; Rueschman, M.; Mariani, S.; Mobley, D.R.; Redline, S. The national sleep research resource: Towards a sleep data commons. J. Am. Med. Inform. Assoc. 2018, 25, 1351–1358.
    15. Quan, S.F.; Howard, B.V.; Iber, C.; Kiley, J.P.; Nieto, F.J.; O’Connor, G.; Rapoport, D.M.; Redline, S.; Robbins, J.; Samet, J.M.; et al. The sleep heart health study: Design, rationale, and methods. Sleep 1997, 20, 1077–1085.
    16. Ichimaru, Y.; Moody, G. Development of the polysomnographic database on CD-ROM. Psychiatry Clin. Neurosci. 1999, 53, 175–177.
    17. Khalighi, S.; Sousa, T.; Santos, J.M.; Nunes, U. ISRUC-Sleep: A comprehensive public dataset for sleep researchers. Comput. Methods Programs Biomed. 2016, 124, 180–192.
    18. O’Reilly, C.; Gosselin, N.; Carrier, J.; Nielsen, T. Montreal archive of sleep studies: An open-access resource for instrument benchmarking and exploratory research. J. Sleep Res. 2014, 23, 628–635.
    19. Li, Q.; Li, Q.C.; Liu, C.; Shashikumar, S.P.; Nemati, S.; Clifford, G.D. Deep learning in the cross-time frequency domain for sleep staging from a single-lead electrocardiogram. Physiol. Meas. 2018, 39, 124005.
    20. Tripathy, R.; Acharya, U.R. Use of features from RR-time series and EEG signals for automated classification of sleep stages in deep neural network framework. Biocybern. Biomed. Eng. 2018, 38, 890–902.
    21. Radha, M.; Fonseca, P.; Moreau, A.; Ross, M.; Cerny, A.; Anderer, P.; Long, X.; Aarts, R.M. Sleep stage classification from heart-rate variability using long short-term memory neural networks. Sci. Rep. 2019, 9, 1–11.
    22. Zhu, T.; Luo, W.; Yu, F. Convolution-and attention-based neural network for automated sleep stage classification. Int. J. Environ. Res. Public Health 2020, 17, 4152.
    23. Qureshi, S.; Karrila, S.; Vanichayobon, S. GACNN SleepTuneNet: A genetic algorithm designing the convolutional neuralnetwork architecture for optimal classification of sleep stages from a single EEG channel. Turk. J. Electr. Eng. Comput. Sci. 2019, 27, 4203–4219.
    24. Yıldırım, Ö.; Baloglu, U.B.; Acharya, U.R. A deep learning model for automated sleep stages classification using PSG signals. Int. J. Environ. Res. Public Health 2019, 16, 599.
    25. Hsu, Y.-L.; Yang, Y.-T.; Wang, J.-S.; Hsu, C.-Y. Automatic sleep stage recurrent neural classifier using energy features of EEG signals. Neurocomputing 2013, 104, 105–114.
    26. Michielli, N.; Acharya, U.R.; Molinari, F. Cascaded LSTM recurrent neural network for automated sleep stage classification using single-channel EEG signals. Comput. Biol. Med. 2019, 106, 71–81.
    27. Wei, L.; Lin, Y.; Wang, J.; Ma, Y. Time-Frequency Convolutional Neural Network for Automatic Sleep Stage Classification Based on Single-Channel EEG. In Proceedings of the 2017 IEEE 29th International Conference on Tools with Artificial Intelligence, Boston, MA, USA, 6–8 November 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 88–95.
    28. Mousavi, S.; Afghah, F.; Acharya, U.R. SleepEEGNet: Automated sleep stage scoring with sequence to sequence deep learning approach. PLoS ONE 2019, 14, e0216456.
    29. Seo, H.; Back, S.; Lee, S.; Park, D.; Kim, T.; Lee, K. Intra- and inter-epoch temporal context network (IITNet) using sub-epoch features for automatic sleep scoring on raw single-channel EEG. Biomed. Signal Process. Control 2020, 61, 102037.
    30. Zhang, X.; Xu, M.; Li, Y.; Su, M.; Xu, Z.; Wang, C.; Kang, D.; Li, H.; Mu, X.; Ding, X.; et al. Automated multi-model deep neural network for sleep stage scoring with unfiltered clinical data. Sleep Breath. 2020, 24, 581–590.
    31. Supratak, A.; Dong, H.; Wu, C.; Guo, Y. DeepSleepNet: A model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1998–2008.
    32. Phan, H.; Andreotti, F.; Cooray, N.; Chén, O.Y.; de Vos, M. Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Trans. Biomed. Eng. 2019, 66, 1285–1296.
    33. Vilamala, A.; Madsen, K.H.; Hansen, L.K. Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring. In Proceedings of the 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), Tokyo, Japan, 25–28 September 2017; pp. 1–6.
    34. Phan, H.; Andreotti, F.; Cooray, N.; Chen, O.Y.; de Vos, M. DNN filter bank improves 1-max pooling CNN for single-channel EEG automatic sleep stage classification. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA, 18–21 July 2018; pp. 453–456.
    35. Xu, M.; Wang, X.; Zhangt, X.; Bin, G.; Jia, Z.; Chen, K. Computation-Efficient Multi-Model Deep Neural Network for Sleep Stage Classification. In Proceedings of the ASSE ’20: 2020 Asia Service Sciences and Software Engineering Conference, Nagoya, Japan, 13–15 May 2020; Association for Computing Machinery: New York, NY, USA, 2020; pp. 1–8.
    36. Wang, Y.; Wu, D. Deep Learning for Sleep Stage Classification. In Proceedings of the 2018 Chinese Automation Congress (CAC), Xi’an, China, 30 November–2 December 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 3833–3838.
    37. Fernandez-Blanco, E.; Rivero, D.; Pazos, A. Convolutional neural networks for sleep stage scoring on a two-channel EEG signal. Soft Comput. 2019, 24, 4067–4079.
    38. Jadhav, P.; Rajguru, G.; Datta, D.; Mukhopadhyay, S. Automatic sleep stage classification using time-frequency images of CWT and transfer learning using convolution neural network. Biocybern. Biomed. Eng. 2020, 40, 494–504.
    39. Tsinalis, O.; Matthews, P.M.; Guo, Y.; Zafeiriou, S. Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks; Imperial College London: London, UK, 2016.
    40. Sokolovsky, M.; Guerrero, F.; Paisarnsrisomsuk, S.; Ruiz, C.; Alvarez, S.A. Deep learning for automated feature discovery and classification of sleep stages. IEEE/ACM Trans. Comput. Biol. Bioinform. 2019, 17.
    41. Dong, H.; Supratak, A.; Pan, W.; Wu, C.; Matthews, P.M.; Guo, Y. Mixed neural network approach for temporal sleep stage classification. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 324–333.
    42. Chambon, S.; Galtier, M.N.; Arnal, P.J.; Wainrib, G.; Gramfort, A. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans. Neural Syst. Rehabil. Eng. 2018, 26, 758–769.
    43. Phan, H.; Andreotti, F.; Cooray, N.; Chén, O.Y.; de Vos, M. SeqSleepNet: End-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 400–410.
    44. Zhang, J.; Yao, R.; Ge, W.; Gao, J. Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG. Comput. Methods Programs Biomed. 2020, 183, 105089.
    45. Zhang, J.; Wu, Y. Complex-valued unsupervised convolutional neural networks for sleep stage classification. Comput. Methods Programs Biomed. 2018, 164, 181–191.
    46. Sors, A.; Bonnet, S.; Mirek, S.; Vercueil, L.; Payen, J.-F. A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomed. Signal Process. Control 2018, 42, 107–114.
    47. Fernández-Varela, I.; Hernández-Pereira, E.; Alvarez-Estevez, D.; Moret-Bonillo, V. A Convolutional Network for Sleep Stages Classification. arXiv 2019, arXiv:1902.05748v1.
    48. Zhang, L.; Fabbri, D.; Upender, R.; Kent, D.T. Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks. Sleep 2019, 42.
    49. Cui, Z.; Zheng, X.; Shao, X.; Cui, L. Automatic sleep stage classification based on convolutional neural network and fine-grained segments. Complexity 2018, 2018, 1–13.
    50. Yang, Y.; Zheng, X.; Yuan, F. A Study on Automatic Sleep Stage Classification Based on CNN-LSTM. In Proceedings of the ICCSE’18: The 3rd International Conference on Crowd Science and Engineering, Singapore, 28–31 July 2018; Association for Computing Machinery: New York, NY, USA, 2018; pp. 1–5.
    51. Yuan, Y.; Jia, K.; Ma, F.; Xun, G.; Wang, Y.; Su, L.; Zhang, A. A hybrid self-attention deep learning framework for multivariate sleep stage classification. BMC Bioinform. 2019, 20, 1–10.
    52. Biswal, S.; Sun, H.; Goparaju, B.; Westover, M.B.; Sun, J.; Bianchi, M.T. Expert-level sleep scoring with deep neural networks. J. Am. Med. Inform. Assoc. 2018, 25, 1643–1650.
    53. Biswal, S.; Kulas, J.; Sun, H.; Goparaju, B.; Westover, M.B.; Bianchi, M.T.; Sun, J. SLEEPNET: Automated Sleep Staging System via Deep Learning. arXiv 2017, arXiv:1707.08262.
    54. Faust, O.; Razaghi, H.; Barika, R.; Ciaccio, E.J.; Acharya, U.R. A review of automated sleep stage scoring based on physiological signals for the new millennia. Comput. Methods Programs Biomed. 2019, 176, 81–91.
    55. Hoshide, S.; Kario, K. Sleep Duration as a risk factor for cardiovascular disease—A review of the recent literature. Curr. Cardiol. Rev. 2010, 6, 54–61.
    56. Woods, S.L.; Froelicher, E.S.S.; Motzer, S.U.; Bridges, S.J. Cardiac Nursing, 5th ed.; Lippincott Williams and Wilkins: London, UK, 2005.
    57. Krieger, J. Breathing during sleep in normal subjects. Clin. Chest Med. 1985, 6, 577–594.
    58. Madsen, P.L.; Schmidt, J.F.; Wildschiodtz, G.; Friberg, L.; Holm, S.; Vorstrup, S.; Lassen, N.A. Cerebral O2 metabolism and cerebral blood flow in humans during deep and rapid-eye-movement sleep. J. Appl. Physiol. 1991, 70, 2597–2601.
    59. Klosh, G.; Kemp, B.; Penzel, T.; Schlogl, A.; Rappelsberger, P.; Trenker, E.; Gruber, G.; Zeithofer, J.; Saletu, B.; Herrmann, W.; et al. The SIESTA project polygraphic and clinical database. IEEE Eng. Med. Boil. Mag. 2001, 20, 51–57.
    1. Please check and comment entries here.