Human control engineering can use several technological means to control human factors, such as workload, attention or vigilance; however, controversies exist about some of them [18], i.e., studies have highlighted that, independent from technology, vigilance can be improved from dieting or fasting [19], or even from chewing gum [20,21]. Second, two kinds of technological supports can influence human cognitive state: passive tools, i.e., human–machine interaction supports, or active ones, which are capable of making decisions and acting accordingly. Examples are listening to music, which may improve concentration [22,23]. Meanwhile, a dedicated decision support system can decrease workload by improving performance [24]. Moreover, due to disengagement from a driving situation under monotonous driving conditions, automation might lead operators to become more fatigued that they would during manual driving conditions [25,26].

A great deal of research has reported on the utility of using physiological, behavioral, auditory and vehicle data to detect the mental state of the driver, such as presence/sleep, drowsiness or cognitive workload, with considerable accuracy [27,28,29]. Diverse parameters can provide information on targeted mental states: physiological measures include signals such as ElectroEncephaloGram (EEG) data, ElectroDermal Activity (EDA), and heart rate and heart rate variability. Behavioral measures include aspects such as head-direction, head-movement, gaze-direction, pose of the superior part of the body, gaze-dispersion, blinking, saccades, PERCLOS, pupil-size, eyelid movement, postural adjustment and nonself-centered gestures. Such data may be combined in multimodal approaches with information on vehicle activity and auditory information. However, existing approaches still present clear limitations, such as with electroencephalography (EEG), which is hardly usable in real contexts due to possible discomfort, an unprovable performance, as well as, in some cases, the high computational cost for calculations, which constrains implementation in real environments [30]. Note also that more traditional behavioral measures used in experimental psychology, such as the secondary task paradigm, have been shown to be quite useful in workload studies [31].

Results from studies on human factors in driver automation based on these techniques are often concerned with questions such as of how users tackle automated driving and transitions between manual and automated control. Most such studies were motivated by the increasing prevalence of automated control in commercial and public transport vehicles, as well as increases in the degree of automation. Moreover, while automated driving significantly reduces workload, this is not the case for Adaptive Cruise Control (ACC) [32]. For instance, a driving simulator and vehicle with eye-tracking measures showed that the time required to resume control of a car is about 15 s, and up to 40 s to stabilize it [33].

Alarms comprising beeps are safer than those comprising sounds with positive or negative emotional connotations [34], and human performances can differ according to the use of interaction means involving hearing or sight [35]. Moreover, interactions with visual or audio-visual displays are more efficient than those with auditory displays only [36]. In this sense, research on multimodal perception is particularly relevant when studying human factors of driver aid systems [37,38].

Other studies have not observed significant impacts of noise or music on human performance [39] and have even concluded that silence is able to increase attention during human disorder recovery conditions [40].

Moreover, the use of decision support systems can generate ambiguous results by leading to dissonances, affordances, contradictions or interferences with safety critical behavior [41,42,43], potentially increasing hypo-vigilance and extending human response time [44]. As an example, the well-known Head-Up Display (HUD) system is useful to display important information without requiring them to move their gaze in several directions, but it is also a mean to focus attention upon a reduced control area [45]. It is, therefore, a tool to prevent accidents, but can also cause problems of focused attention.

Neuropsychological studies generally use sensors connected to the brain to assess neural activities related to cognitive processes, such as perception or problem solving. In this context, eye trackers have been demonstrated to be useful for the study of visual attention or workload via parameters such as closure percentage, blink frequency, fixation duration, etc. [46,47,48,49]. Indeed, the pupil diameter increases with the increasing demand of the performed task and higher the cognitive loads [50], while an increase of physical demand does the opposite [51], as do external cues, such as variations of ambient light, use of drugs or strong emotions. Facial recognition is also incapable of detecting emotional dissonances between expressed and felt emotions. Moreover, eye blink frequency reduces as workload increases [52,53], but it increases when a secondary task is required [54,55].

Eye-trackers can be useful to analyze overt or covert attention: when a subject looks at a point on a scene, the analysis of the corresponding eye movement supposes that the attention is focused on this point, whereas attention can also focus on other points without any eye movement [56].

Variations in heartbeat frequently correspond to variations in the level of the workload, stress or emotion [57,58,59,60], but a new hypothesis considers that perceptive ability can depend on the synchronization between frequency of dynamic events and heart beats. Recent studies have demonstrated that flashing alarms synchronized with heart rate could reduce the solicitation of the insula, i.e., the part of brain dedicated to perception, and the ability to detect it correctly [61,62]. This performance-shaping factor based on the synchronization of dynamic events with heartbeats is relevant for human error analysis.

The development of future smart tools to support driving tasks has to consider extended abilities, such as the ability:

• To cooperate with and learn from others [63,64,65];
• To explain results in a pedagogical way [17];
• To discover and control dissonances between users and support tools [41,42].

Significant advances for the prediction of driver drowsiness and workload have been made in association with the use of more sophisticated features of physiological signals, as well as from the application of increasingly sophisticated machine learning models, although extrapolation of such to the context of commercial pilots has not yet been attempted. Some approaches have been based on EDA signal decomposition into tonic and phasic components [66], extraction of features in time, frequency, and time-frequency (wavelet based) domains [67], or the use of signal-entropy related features [68].

Moreover, regarding machine-learning models, while the most widely used approach is the support vector machine, artificial neural networks, such as convolutional neural networks, seems to provide better performance for the detection of drowsiness and workload [69,70,71].

The combination of such approaches with multimodal data fusion has been shown to provide a very high degree of accuracy for drowsiness detection [72].

Such approaches are applicable to overcome some of the current limitations in the detection in pilots of drowsiness and mental workload. For instance, the high accuracy accomplished with only a subset of the signals suggests that various predictive models of drowsiness and workload could be trained based on different subsets of features, thereby helping to make the system useful, even when some specific features are not momentarily available (e.g., due to occlusion of the eyes or head). Recent advances can also help in the implementation of detection systems with lower computational cost, such as efficient techniques for signal filtering [73] and feature-selection methods to reduce model dimensionality and complexity [74].

Many technologies to control devices by gestures are already on the market. An extended, though not comprehensive, summary of them is presented below.

DEPTHSENSE CARLIB, by Sony, aims to control infotainement by hand movement [75].

EYEDRIVE GESTURE CONTROL by EyeLights is an infrared motion sensor that recognizes simple hand gestures while driving in order to control in-vehicle devices [76].

HAPTIX by Haptix Touch is a webcam-based environment to recognize any classical hand movement and build a virtual mouse to control screen interface [77].

KINECT by Microsoft is a web-cam based device that can capture motion and control devices with body or hand movements [78,79].

LEAP MOTION by Leap Motion Inc. (now UltraHaptics) is an environment for hand movement recognition dedicated to virtual reality. Movement detection is by infrared light, while micro-cameras detect the hands or other objects in 3D [80].

MYO BRACELET by Thalmic Labs proposes an armband to control interfaces with hand or finger movement detected via the electrical activities of activated muscles [74,81,82].

SOLI by Google comprises a mini-radar which is capable of identifying movements, from fingers to the whole body [83,84].

SWIPE by FIBARO is dedicated to home automation; it is controlled via hand motions in front of a simple, contactless tablet [85].

XPERIA TOUCH DEVICE by Sony is a smartphone application for gesture control which is capable of tracking proximate hand gesture via the phone camera [86].

Table 1 summarizes a Strengths Weaknesses Opportunities and Threats (SWOT) analysis of three of systems defined above: KINECT, LEAP MOTION and MYO BRACELET developed starting from the results of similar studies [87].