Detecting Deceptive Behaviours: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Dean Liu and Version 3 by Dean Liu.

Interest in detecting deceptive behaviours by various application fields, such as security systems, political debates, advanced intelligent user interfaces, etc., makes automatic deception detection an active research topic. This interest has stimulated the development of many deception-detection methods in the literature in recent years.

  • deception detection
  • facial cues
  • dataset

1. Introduction

Lying is a complex social activity by someone (i.e., the deceiver) aimed at causing a specific behaviour in another person (i.e., the deceived) by making their view of the situation more congruent with the view and behaviour of the deceived that usually occurs, according to the deceiver’s knowledge [1]. Deception may destroy the relationship and hamper communication, leading to negative consequences [2]. Therefore, deception detection has been an investigated research topic for decades, and the development of tools and methods for detecting deceptive behaviours has become an urgent need for society. However, despite a rich corpus of deception research, detecting deceptive behaviours is still a very challenging task, mainly because human beings do not have good lie-detection abilities. A study by Bond and DePaulo [3] quantifies the human accuracy of discriminating between lies and truths at 54% on average, in other words just slightly above a random guess. This percentage is also confirmed by the Hartwig’s [4] more recent study. In response to the necessity to improve this accuracy rate, researchers have long been trying to decode human behaviour in an attempt to discover deceptive cues. What researchers consider “deceptive” is human behaviour in which the deceiver intentionally acts to make the deceived believe in something the deceiver considers false [5]. This is a conscious and deliberated act, opposed to the unconscious (non-deceptive) behaviour in which a person provides false information believed to be true.
To discover deceptive cues, different methods have been proposed by scientists from different disciplines, ranging from psychologists and physiologists to technologists. For instance, methods based on the psycho-physiological detection of deception that uses mainly psychological tests and physiological monitoring have been applied [6][7]. A positive contribution is also given by technologists that developed methods for deceptive detection based on technological tools. Researchers focused on the technological field, in which a considerable number of methods for detecting verbal (explicit) and non-verbal (implicit) cues of deception have been developed [8]. Explicit cues are categorised according to their nature (i.e., visual, verbal, or paralinguistic) [9], while implicit cues involve facial expressions, body movements/posture, eye contact, and hand movements [10].
Detecting deceptive behaviours from explicit cues makes deception harder because explicit cues are easier to manipulate through carefully chosen language and wording, while implicit cues, even if also subject to manipulation, are more spontaneous and challenging to control consciously [11]. On the other hand, the interpretation of implicit cues tends to be more subjective and context-dependent, requiring a deeper understanding of individual differences and cultural norms, while explicit cues are typically more straightforward and easier to analyse in isolation. Finally, detection deception using implicit cues requires collecting data through video recordings or specialised sensors to capture non-verbal behaviours, which is more challenging compared to collecting text data for explicit cues.
Considering the above-mentioned differences, the solutions provided in the literature for detecting deceptive behaviours have different characteristics if relying on explicit or implicit cues. The systematic literature review provided deals with implicit cues and, in particular, focuses on a specific type of implicit cues, i.e., facial cues. Specifically, the most relevant deception-detection systems focusing on facial cues have been described and classified considering the methods used in the main steps of the facial-deception-detection process (i.e., video pre-processing, facial feature extraction, and decision making). The choice to focus on facial cues is justified by the results of several existing studies [12][13] arguing that facial cues have an important role in detecting deceptive behaviours due to their involuntary aspects.

2. Surveying Automated Deception Detection from Videos

Automated deception detection from videos is a challenging task that can find applications in many real-world scenarios, including airport security screening, job interviews, court trials, and personal credit risk assessment [14].
Various surveys of automated deception-detection methods from videos have been developed in the literature, classifying the methods according to different dimensions. In particular, recent surveys (see Table 1) have provided a very broad overview of deception-detection methods considering monomodal and multimodal features. The survey developed by Constâncio et al. [15] provides a systematic literature review of deception detection based on machine-learning methods, underlying the techniques and approaches applied, the difficulties, the kind of data, and the performance levels achieved. In this survey, verbal and non-verbal cues, such as facial expressions, gestures, and prosodic and linguistic features, have been considered. The 81 surveyed papers are classified according to the type of extracted features (emotional, psychological, and facial) and the kinds of machine-learning algorithms. Analogously, the survey in [16] gives an overview of automated deception detection through machine-intelligence-based techniques by providing a critical analysis of the existing tools and available datasets. The authors focused on deception detection through text, speech, and video data analysis by classifying the 100 surveyed papers according to both the research domain (i.e., psychological, professional, and computational) and the type of extracted features (verbal, non-verbal, and multimodal). A further survey focused on monomodal features (speech cues) is proposed by Herchonvicz and de Santiago [17], in which deep-learning-based techniques, available datasets, and metrics extracted from the surveyed papers are discussed.
Table 1. Comparative analysis of surveys.
Reference Cues Techniques Datasets Steps of the Deception Detection Process (Figure 1)
Constâncio et al., 2023 [15] Verbal and non-verbal cues Machine-learning methods Not surveyed Step 2 and Step 3
Alaskar et al., 2023 [16] Verbal and non-verbal cues Machine-intelligence-based techniques Surveyed Step 2 and Step 3
Herchonvicz and de Santiago, 2021 [17] Verbal cues Deep-learning-based techniques Surveyed Step 2 and Step 3
Our survey Non-verbal cues Machine-learning methods Surveyed Step 1, Step 2, and Step 3
Applsci 13 09188 g001
Figure 1. The workflow of the deception detection process from videos followed in the review.
Different from existing surveys, this work focuses on deception detection methods specifically based on facial cues and proposes a classification of these methods based on the main steps of the facial deception detection process (video pre-processing, facial feature extraction, and decision making), as proposed by Thannoon et al. [18]. On the contrary, the existing surveys considered only the second and third steps of this process.
A general workflow of the followed deception detection process that used video datasets is provided in Figure 1.
During the video pre-processing step, the video is analysed by a face detector, which bounds the box containing the face, and by a facial-landmark-tracking tool, which locates and tracks the facial landmarks. In the facial-feature-extraction step, the extraction of a set of facial features for recognizing the facial cues meaningful to deception is carried out. Finally, the classification of the extracted features into truthful or deceptive behaviour is performed in the decision-making step. According to these steps, the studies included in the review are analysed to extract (i) the methods used for video pre-processing, (ii) the facial extracted features, (iii) the decision-making algorithms, and (iv) the datasets used for the evaluation.

References

  1. Jakubowska, J.; Białecka-Pikul, M. A new model of the development of deception: Disentangling the role of false-belief understanding in deceptive ability. Soc. Dev. 2020, 29, 21–40.
  2. Feng, Y.; Hung, S.; Hsieh, P. Detecting spontaneous deception in the brain. Hum. Brain Mapp. 2022, 43, 3257–3269.
  3. Bond, C.F., Jr.; DePaulo, B.M. Accuracy of deception judgments. Personal. Soc. Psychol. Rev. 2006, 10, 214–234.
  4. Hartwig, M.; Voss, J.A.; Brimbal, L.; Wallace, D.B. Investment Professionals’ Ability to Detect Deception: Accuracy, Bias and Metacognitive Realism. J. Behav. Financ. 2017, 18, 1–13.
  5. Zuckerman, M.; DePaulo, B.M.; Rosenthal, R. Verbal and Nonverbal Communication of Deception. In Advances in Experimental Social Psychology; Academic Press Inc.: Cambridge, MA, USA, 1981.
  6. Viglione, D.J.; Giromini, L.; Landis, P. The Development of the Inventory of Problems: A Brief Self-Administered Measure for Discriminating Bona Fide From Feigned Psychiatric and Cognitive Complaints. J. Pers. Assess. 2016, 99, 534–544.
  7. Vance, N.; Speth, J.; Khan, S.; Czajka, A.; Bowyer, K.W.; Wright, D.; Flynn, P. Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results. IEEE Trans. Biom. Behav. Identity Sci. 2022, 4, 522–532.
  8. Randhavane, T.; Bhattacharya, U.; Kapsaskis, K.; Gray, K.; Bera, A.; Manocha, D. The liar’s walk: Detecting deception with gait and gesture. arXiv 2019, arXiv:1912.06874.
  9. Malik, J.S.; Pang, G.; Hengel, A.V.D. Deep learning for hate speech detection: A comparative study. arXiv 2022, arXiv:2202.09517.
  10. Ogale, N.A. A Survey of Techniques for Human Detection from Video; University of Maryland: College Park, MD, USA, 2006; Volume 125, p. 19.
  11. Ekman, P. Darwin, deception, and facial expression. Ann. N. Y. Acad. Sci. 2003, 1000, 205–221.
  12. Matsumoto, D.; Hwang, H.C. Microexpressions Differentiate Truths from Lies About Future Malicious Intent. Front. Psychol. 2018, 9, 2545.
  13. Darwin, C. The Expression of the Emotions in Man and Animals; J. Murray: London, UK, 1872.
  14. Ding, M.; Zhao, A.; Lu, Z.; Xiang, T.; Wen, J.R. Face-focused cross-stream network for deception detection in videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019.
  15. Constâncio, A.S.; Tsunoda, D.F.; Silva, H.d.F.N.; da Silveira, J.M.; Carvalho, D.R. Deception detection with machine learning: A systematic review and statistical analysis. PLoS ONE 2023, 18, e0281323.
  16. Alaskar, H.; Sbaï, Z.; Khan, W.; Hussain, A.; Alrawais, A. Intelligent techniques for deception detection: A survey and critical study. Soft Comput. 2023, 27, 3581–3600.
  17. Herchonvicz, A.L.; de Santiago, R. Deep Neural Network Architectures for Speech Deception Detection: A Brief Survey. In Progress in Artificial Intelligence, Proceedings of the 20th EPIA Conference on Artificial Intelligence, EPIA 2021, Virtual Event, 7–9 September 2021; Springer International Publishing: Cham, Switzerland, 2021; pp. 301–312.
  18. Thannoon, H.H.; Ali, W.H.; Hashim, I.A. Detection of Deception Using Facial Expressions Based on Different Classification Algorithms. In Proceedings of the Third Scientific Conference of Electrical Engineering (SCEE), IEEE, Baghdad, Iraq, 19–20 December 2019; pp. 51–56.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback