Continuous advancements in computing technology and artificial intelligence have led to improvements in driver monitoring systems. Numerous experimental studies have collected real driver drowsiness data and applied various artificial intelligence algorithms and feature combinations with the goal of significantly enhancing the performance of these systems in real-time.
1. Introduction
Based on 2017 police and hospital reports, the National Highway Traffic Safety Administration (NHTSA) identified 91,000 car accidents as being caused by drowsy drivers. These accidents resulted in 50,000 injuries. In 2019, 697 fatalities involved a drowsy driver. However, NHTSA admits that it is hard to determine the precise number of drowsy-driving accidents, injuries, or deaths and that the reported numbers are underestimates
[1]. For example, a study by the American Automobile Association’s foundation for traffic safety estimated that more than 320,000 drowsy driving accidents happen each year, including 6400 fatal crashes
[2]. The high numbers indicate that drowsy driving is a serious concern that needs to be addressed to mitigate its impact.
Drowsiness refers to sleepiness, often in inappropriate situations
[3]. Although the state of drowsiness may only last for a few minutes, its consequences can be disastrous. The reason for entering such a state is usually attributed to fatigue, which diminishes attention and alertness levels
[4]. Drowsiness may happen either by driving for long distances without enough sleep or driving at a time when the driver would typically be asleep
[5]. In such cases, the main problem is the drowsy driver’s lack of concentration, resulting in a delayed response to any event on the road
[6].
Fortunately, it is possible to detect driver drowsiness in its early stages and alarm the driver to avoid any potential accident. Drowsy drivers exhibit various signs, which include repeated yawning, frequent eye closure, and repeatedly departing street lanes
[6]. In fact, driver drowsiness detection (DDD) techniques have been researched intensively in recent years
[7][8][9][10][11][12][13]. Researchers have proposed various measures to detect these drowsiness signs as early as possible, in order to avoid accidents. These measures can be divided into four main categories: firstly, image-based measures that are obtained using a camera to analyze the driver’s movements and facial expressions; secondly, biological-based measures that relate to the driver’s bio-signals and can be recorded by placing special sensors on the driver’s body; thirdly, vehicle-based measures, which depend on monitoring the behavior and movement of the vehicle; finally, hybrid-based measures, using two or more measures. According to the literature, in 2019, Ramzan et al.
[9] presented a comprehensive analysis for the existing DDD methods, as well as a detailed analysis for the commonly used classification techniques in this sector. Ramzan et al. classified the DDD techniques into three categories: behavioral, physiological, and vehicular parameter-based techniques. Then, they reviewed the top supervised learning techniques used in detecting drowsiness. In the end, they discussed the pros and cons of the three DDD in a comparative study. On the other hand, Sikander and Anwar
[10] presented an in-depth review of the recent advancements in the field of driver fatigue detection. Here, the DDD methods were categorized into five groups, depending on the extracted fatigue features, including physical features, vehicular features, biological features, subjective reporting, and hybrid features. Furthermore, the fatigue effect on driving performance was discussed, along with the existing commercial products for fatigue detection available on the market. Additionally, Dong et al. presented a review of driver inattention monitoring technologies. Inattention consists of distraction and fatigue
[11]. Dong et al. summarized the detection measure into five groups, similar to Sikander and Anwar’s work
[10]. In their review, Dong et al. introduced the concept of driver inattention and its effect on driving performance. Additionally, they covered some of the commercial products related to inattention detection, along with a detailed review of previous research on inattention detection.
2. Drowsiness Detection Measures
In order to detect the different stages of drowsiness, researchers have studied driver responses and vehicle driving patterns. This section provides an overview of the four widely used measures for DDD. The diagram in Figure 1 illustrates all the currently used measures for classifying driver drowsiness levels. Two of these measures are observed in the drivers themselves: image- and biological-based. The third measure is extracted from the car itself and referred to as the vehicle-based measure. The fourth measure considered is the hybrid measure, which combines at least two of the previously mentioned ones.
Figure 1. Driver drowsiness detection measures.
Figure 2 illustrates a DDD system’s general block diagram and data flow that can employ any of the four measures mentioned above. Initially, data are captured using a suitable sensing device; then, the target features are extracted from the captured signals. This step is essential because it simplifies the system input by discarding irrelevant information and extracting useful ones. Next, some systems may employ feature transformation or dimensionality reduction, in order to project the data in another domain, where it is easier to analyze or reduce the computational load. The fourth step selects the features that best correlate to drowsiness, using different feature selection algorithms, such as backward selection or wrapper feature selection methods. After that, machine learning (ML) or deep learning is utilized to generate a model in the training phase that is used to classify the driver’s status. The trained model is used in the testing phase to detect the driver’s drowsiness level and, if required, take action, such as activating an alarm or alerting the driver to take a break.

Figure 2. Driver drowsiness detection systems data flow.
Various metrics have been used to evaluate the ability of the system to detect drowsy subjects. These include accuracy, precision, and sensitivity. The equations for three metrics are listed below (1)–(3)
[14][15].
(1)
(2)
(3)
2.1. Image-Based Measures
Some drowsiness signs are visible and can be recorded by cameras or visual sensors. They include the driver’s facial expressions and movements, especially the head movements. The literature refers to these signs as visual
[8] or image-based measures
[7]. Herein refers to them as image-based measures to highlight that these measures usually lead to features extracted from images or videos. Additionally, it is important to note here that image-based measures are a subcategory of the physical
[10] or behavioral measures
[9]. Physical and behavioral measures refer to the body movements captured either from videos or using motion sensors, such as a gyroscope and accelerometer
[16][17].
Image-based DDD systems can be broadly categorized into three techniques, based on whether movements of the mouth, head, or eyes are observed. Table 1 lists some of the image-based measures.
Table 1. Some of the image-based measures.
2.2. Biological-Based Measures
Many biological signals have been used to detect the driver’s drowsiness, such as brain activity, heart rate, breathing rate, pulse rate, and body temperature signals
[10]. These biological signals, also known as physiological measures
[9], are proven to be more accurate and reliable for detecting drowsiness. This accuracy is due to their ability to capture early biological changes that may appear, in the case of drowsiness, thus alerting the driver before any physical drowsiness signs appear. The most commonly used biological measures in literature are listed in
Table 2.
Table 2. Some biological-based measures.
2.3. Vehicle-Based Measures
This method depends on tracing and analyzing driving patterns. Every driver forms a unique driving pattern. Thus, the driving patterns of a drowsy driver can be easily distinguished from those of an alert driver. According to Pratama et al.
[8], vehicular-based measures are the least investigated methods, due to the difficulty of precisely determining drowsy driving state features. Thus, many researchers combine this measure with image-based or biological measures
[29][30][31]. The two most common detected vehicle-based measures, used to identify driver drowsiness, are steering wheel angle (SWA) and lane departure.
Table 3 provides a list of DDD systems based on vehicle measures.
Table 3. Vehicle-based drowsiness detection systems.
2.4. Hybrid-Based Measures
A hybrid DDD system employs a combination of image-, biological-, and vehicle-based measures to extract drowsiness features, with the aim of producing a more robust, accurate, and reliable DDD system. Table 4 shows a list of some of the recently proposed hybrid DDD systems.
Table 4. Hybrid-based drowsiness detection systems.
3. Conclusions
Over the past decade, the drowsiness detection field has experienced significant enhancements, due to technological advancements in IoT, sensor miniaturization, and artificial intelligence. Herein has presented a detailed and up-to-date conclusion of the driver drowsiness detection systems that have been implemented in the last ten years. It has described the four main approaches followed in designing DDD systems and categorized them based on the type of drowsiness indicative parameters employed. These four categories are image-, biological-, vehicle-, and hybrid-based systems.
5G networks are expected to play a prominent role in enhancing DDD systems. With 5G connectivity, future DDD systems will be based on real driving scenarios. The data will be obtained from various drivers in actual vehicles, where factors such as ambient light, road surface vibrations, and individual differences among drivers are considered. The use of 5G connectivity will also enable the use of multi-access edge computing power for deep learning, resulting in highly accurate real-time decisions. Vehicles are expected to operate as members of Internet of vehicle networks, enabling the network to warn the drowsy driver, take control of the car (if needed), and contact neighboring vehicles in the network to alert them about the weary driver. These technologies will lead to safer roads and pave the way towards realizing smart cities.
Herein conclude by emphasizing that DDD technology has enormous market potential. Many car manufacturers, such as Toyota and Nissan, have recently installed or upgraded driver assistance devices in their products. The artificial intelligence and deep learning fields are developing tremendously. Soon, the DDD systems will most likely evolve, enabling the formation of smart cities.
Abbreviations
The nomenclature abbreviations, shown in Nomenclature, were used in this manuscript.