Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1
Gregor Strle
 -- 1727 2023-08-10 16:22:29

Video Upload Options

Do you have a full video?
Send video materials Upload full video

Confirm

Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Strle, G.; Košir, A.; Burnik, U. Physiological Measurement of Advertising Engagement. Encyclopedia. Available online: https://encyclopedia.pub/entry/47914 (accessed on 08 July 2024).
Strle G, Košir A, Burnik U. Physiological Measurement of Advertising Engagement. Encyclopedia. Available at: https://encyclopedia.pub/entry/47914. Accessed July 08, 2024.
Strle, Gregor, Andrej Košir, Urban Burnik. "Physiological Measurement of Advertising Engagement" Encyclopedia, https://encyclopedia.pub/entry/47914 (accessed July 08, 2024).
Strle, G., Košir, A., & Burnik, U. (2023, August 10). Physiological Measurement of Advertising Engagement. In Encyclopedia. https://encyclopedia.pub/entry/47914
Strle, Gregor, et al. "Physiological Measurement of Advertising Engagement." Encyclopedia. Web. 10 August, 2023.
Physiological Measurement of Advertising Engagement
Edit
This entry is adapted from the peer-reviewed paper 10.3390/s23156916

Advertising evokes emotional responses and triggers cognitive processes in consumers. This, in turn, influences individuals’ physiological responses and can provide new insights into consumer behavior. Owing to the steady development of wearable devices with physiological sensors, physiological signals may provide a way to directly and continuously measure the effects of ad exposure and thus better understand consumer behavior.

physiology affect engagement advertisement user modeling classification machine learning

1. Advertisement (Ad) Engagement

Engagement is a multidimensional concept, covering the affective, behavioral, and cognitive aspects of individuals’ interactions with advertisements and brands [1][2][3][4][5][6]. According to Hollebeek et al, customer engagement is “a customer’s motivationally driven, volitional investment of operant resources (including cognitive, emotional, behavioral, and social knowledge and skills), and operand resources (e.g., equipment) into brand interactions” [7]. Higher consumer engagement leads to more attention, improves brand memorability and attitude [8][9], and can positively influence consumer purchase intentions [1][5]. Therefore, consumer engagement is an important metric to quantify brand exposure and advertising effectiveness.
An analysis of over 200 media exposure studies conducted by [10] found that there is no one-size-fits-all solution for defining and measuring engagement because consumer engagement is based on context-specific experiences and attitudes. In the digital advertising landscape, novel approaches to measuring ad engagement that focus on the affective and behavioral aspects of consumers’ interactions with ads are increasingly being considered [1][3][11].
For example, [4] examined engagement in terms of behavioral manifestations in the consumer–brand relationship, while [3][11] focused on affective responses to ads. The latter are key to understanding ad engagement, as elevated arousal has been shown to increase engagement behavior [12]. To this end, Eijlers et al. [11] examined how arousal is represented in the brain in response to ads, measured by notability and attitude toward ads. An electroencephalogram (EEG) was recorded for 23 female participants (age: 18–30 years). Their results showed that arousal was positively associated with notability, but attitude toward the ads was negatively associated with arousal in response to the ads. The behavioral and social dimensions of engagement were conceptualized using the User Engagement Scale (UES) developed in [2]. UES measures affect, aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability [2].
Most of these studies have examined ad engagement using self-reported data alone. This has several drawbacks: self-reporting is time-consuming and prone to rater bias [13]. Also, it cannot measure ad engagement directly. In the context of ad-supported video streaming, self-reports are difficult to implement because they distract consumers from immersive experience [8]. All of these shortcomings were the main motivations for the alternative presented here: to investigate direct and unobtrusive measures of ad engagement.

2. Physiological Measurement of Engagement

More direct methods of measuring consumer engagement are being explored, with the goal of measuring ad engagement unobtrusively and in real time, without interrupting the streaming experience.
One approach is to measure ad engagement using psychophysiological signals associated with emotional arousal and cognitive processes. Physiological signals have been studied in detail as potential indicators of a user’s emotional arousal and cognitive load in various human–computer interaction situations. Signals such as electroencephalography, cardiac (heart rate and heart rate variability), electrodermal activity, skin temperature, respiratory activity, and eye measurements (e.g., gaze, pupil dilation, eye blinking) have all been found to be associated with emotional arousal and cognitive load [14][15][16][17].
A study conducted by [18] demonstrated that common physiological signals such as heart rate, skin temperature, respiratory rate, oxygen saturation, blood pressure, and electrocardiogram (ECG) data can be used to measure engagement. Another study [19] examined engagement during video and audio narration using wrist sensors for heart rate variability, electrodermal activity, and body temperature. There was a significant physiological response to all three measures. Ayres et al. conducted a comprehensive examination of physiological data on intrinsic cognitive load, including pupil dilation, blink rate, fixation, heart rate, heart rate variability, electrodermal measurements, respiratory measurements, functional near-infrared spectroscopy, electroencephalography, and functional magnetic resonance imaging [20]. They found that the blink rate, heart rate, pupil dilation, and alpha waves were the most sensitive physiological measurements.
Pupil dilation and heart rate have been studied extensively as physiological measures of arousal and identified as potential indicators of cognitive performance in participants [15][17][21][22]. In [23], it was suggested that personalized advertising systems based on instantaneous measurement of heart rate variability could be used in future advertising strategies. According to [21], heart rate variability (HRV) can be a reliable and cost-effective source of data for neurophysiological and psychophysiological studies, when appropriate acquisition protocols and well-developed indices are available. Schaffer et al. [24] studied the complexity of cardiac oscillations in their evaluation of methods for measuring HRV in the time and frequency domains, using nonlinear metrics.
Pupil dilation can be used to assess cognitive load. According to [25], an increase in task demands leads to an increase in pupil dilation in the cognitive control domains of updating, switching, and inhibition. However, the study did not provide a clear explanation of the relationship between pupil dilation and performance. Since the early 1970s, researchers have used pupil dilation in studies of advertising effectiveness [26]. As pupil dilation is sensitive to variations in brightness, it is difficult to use pupil dilation to measure emotional arousal during video viewing. In [27], a linear model that predicts a viewer’s pupil diameter based only on the incident light intensity was presented. The model can be used to subtract the effects of brightness to determine the subjects’ emotional arousal as a function of the scene viewed.
The skin temperature signal represents changes in blood flow controlled by autonomic nervous system activity. Several studies have found that skin temperature is an effective indicator of human activity and other psychophysiological states, including affect, stress, attention, cognitive load, and drowsiness. Ioannou et al. reviewed 23 experimental procedures that used functional infrared thermal imaging to investigate this effect [28]. A major advantage of this approach is its unobtrusiveness. In particular, the temperature of the facial skin has been the subject of several studies because the face is constantly exposed and can be measured remotely using infrared thermography [28][29][30][31].
For example, [31] investigated the use of skin temperature to estimate resting blood pressure by separating acute stress variations using a multiple regression analysis. They reported that the trained model could accurately estimate resting blood pressure from facial thermal images with a root mean square error of 9.90 mmHg [31]. Another study [29] examined the utility of infrared thermal imaging for stress research. They compared thermal images to established stress indicators (heart rate, heart rate variability, finger temperature, alpha-amylase, and cortisol) in 15 participants who underwent two laboratory stress tests: the Cold Pressor Test and the Trier Social Stress Test. Their results showed that thermal imprints were sensitive to changes in both tests and that thermal imprints correlated with stress-induced mood changes, whereas established stress markers did not [29]. In a study by [30], facial skin temperature was also used. Their results showed that “skin temperature changes have both reproducible and individual response characteristics to drowsiness” [30] (p. 875). Consequently, a convolutional neural network (CNN) model was based on the distributions of facial skin temperature and trained on feature maps and individual models for each subject. The authors reported that the discrimination rate calculated using the CNN was at least 20% higher than that obtained using conventional methods.

3. Modeling Engagement with Machine Learning

Several novel approaches have been developed to directly predict engagement using machine learning methods based on physiological signals.
To this end, DeepWalk, a graph-embedding model, was developed by [32] to predict the video engagement of an ad to detect ad fraud. This model can detect fake videos and fraud patterns in videos containing well-known brands. More generally, Ref. [33] proposed an automatic approach for processing and evaluating learner engagement. They developed a prediction model for context-agnostic engagement based on the video features of the learner content and engagement signals. Research conducted by [6] on YouTube review videos (with a total duration of 600 h) identified features and patterns relevant to emotion (valence and arousal) and trustworthiness as the underlying dimensions of engagement. Several indicators of user engagement, including views, the ratio of likes to dislikes, and sentiment in comments, served as the ground truth. A study by [34] defined a set of video metrics (including video quality, time, and average percentage of videos viewed) to model the relative engagement based on 5.3 million YouTube videos. The results show that video engagement metrics are stable over time, with most of the variance explained by video context, topics, and channel information. The study also found that the time spent watching a video is a better indicator of engagement than the number of views commonly used in recent ad engagement studies.
A study by [35] examined user engagement in the context of emotional arousal (distinguishing between relaxing or exciting stimuli) during multimedia exposure and created a model based on the patterns of physiological responses of five participants to multimedia stimuli. Using machine learning, they created a predictive model of engagement based on the physiological responses to audiovisual materials. The authors performed emotional arousal classification based on affect and physiological signals from the GSR, ECG, EOG, EEG, and PPG, extracting 98 features from the five signals. Affect recognition, emotion recognition, and classification methods were used. This study found that the patterns of physiological responses to each multimedia stimulus were effective in classifying the stimulus types. The authors reported that arousal classification was achieved with 88.9% accuracy and an average recall of 83.3% for models validated using leave-one-subject-out cross-validation. Although the reported performance is high, the selected metrics of precision and accuracy are questionable because of the high class imbalance.
A more recent study by [36] explored the utilization of physiological signals (heart rate, electrodermal activity, pupil dilation, and skin temperature) and affective states in predicting viewer emotional engagement in ads. Drawing data from 50 young adults and using gradient boosting classifiers, the study found that the fusion of skin temperature, perceived tiredness, and emotional valence provided the most accurate prediction of engagement (average ROC AUC = 0.69 (0.07); best ROC AUC = 0.92). The main finding of this study is that the emotional engagement in ads can be modeled with physiological signals alone, retaining comparable performance to the models based on combining physiological signals and affective states. This is particularly important in cases where continuous measurement of engagement is desired and physiological signals must be evaluated in near real time.
The main advantage of combining physiological signals with machine learning is that it allows for scalable, automatic, and continuous assessment of engagement without being intrusive or requiring explicit user feedback. However, research on the use of machine learning and physiology to model ad engagement remains limited.

References

  1. Araujo, T.; Copulsky, J.R.; Hayes, J.L.; Kim, S.J.; Srivastava, J. From Purchasing Exposure to Fostering Engagement: Brand–Consumer Experiences in the Emerging Computational Advertising Landscape. J. Advert. 2020, 49, 428–445.
  2. O’Brien, H.L.; Cairns, P.; Hall, M. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. Int. J. Hum.-Comput. Stud. 2018, 112, 28–39.
  3. Nijholt, A.; Vinciarelli, A. Measuring engagement: Affective and social cues in interactive media. In Proceedings of the 8th International Conference on Methods and Techniques in Behavioral Research, Measuring Behavior, Utrecht, The Netherlands, 28–31 August 2012; pp. 72–74.
  4. Hollebeek, L.; Glynn, M.; Brodie, R. Consumer Brand Engagement in Social Media: Conceptualization, Scale Development and Validation. J. Interact. Mark. 2014, 28, 149–165.
  5. Calder, B.J.; Isaac, M.S.; Malthouse, E.C. How to Capture Consumer Experiences: A Context-Specific Approach To Measuring Engagement. J. Advert. Res. 2016, 56, 39–52.
  6. Stappen, L.; Baird, A.; Lienhart, M.; Bätz, A.; Schuller, B. An Estimation of Online Video User Engagement From Features of Time- and Value-Continuous, Dimensional Emotions. Front. Comput. Sci. 2022, 4, 773154.
  7. Hollebeek, L.D.; Srivastava, R.K.; Chen, T. S-D logic–informed customer engagement: Integrative framework, revised fundamental propositions, and application to CRM. J. Acad. Mark. Sci. 2019, 47, 161–185.
  8. Niederdeppe, J. Meeting the Challenge of Measuring Communication Exposure in the Digital Age. Commun. Methods Meas. 2016, 10, 170–172.
  9. Shen, W.; Bai, H.; Ball, L.J.; Yuan, Y.; Wang, M. What makes creative advertisements memorable? The role of insight. Psychol. Res. 2021, 85, 2538–2552.
  10. de Vreese, C.H.; Neijens, P. Measuring Media Exposure in a Changing Communications Environment. Commun. Methods Meas. 2016, 10, 69–80.
  11. Eijlers, E.; Boksem, M.A.S.; Smidts, A. Measuring Neural Arousal for Advertisements and Its Relationship With Advertising Success. Front. Neurosci. 2020, 14, 736.
  12. Schreiner, M.; Fischer, T.; Riedl, R. Impact of content characteristics and emotion on behavioral engagement in social media: Literature review and research agenda. Electron. Commer. Res. 2021, 21, 329–345.
  13. Hoyt, W.T. Rater bias in psychological research: When is it a problem and what can we do about it? Psychol. Methods 2000, 5, 64.
  14. Camm, A.J.; Malik, M.; Bigger, J.T.; Breithardt, G.; Cerutti, S.; Cohen, R.J.; Coumel, P.; Fallen, E.L.; Kennedy, H.L.; Kleiger, R.E.; et al. Heart rate variability: Standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation 1996, 93, 1043–1065.
  15. Iqbal, S.T.; Zheng, X.S.; Bailey, B.P. Task-evoked pupillary response to mental workload in human-computer interaction. In Proceedings of the CHI’04 Extended Abstracts on Human Factors in Computing Systems, Vienna, Austria, 24–29 April 2004; pp. 1477–1480.
  16. Jerritta, S.; Murugappan, M.; Nagarajan, R.; Wan, K. Physiological signals based human emotion Recognition: A review. In Proceedings of the 2011 IEEE 7th International Colloquium on Signal Processing and its Applications, Penang, Malaysia, 4–6 March 2011; pp. 410–415.
  17. Pomplun, M.; Sunkara, S. Pupil dilation as an indicator of cognitive workload in human-computer interaction. In Human-Centered Computing; CRC Press: Boca Raton, FL, USA, 2019; pp. 542–546.
  18. Bustos-López, M.; Cruz-Ramírez, N.; Guerra-Hernández, A.; Sánchez-Morales, L.N.; Cruz-Ramos, N.A.; Alor-Hernández, G. Wearables for Engagement Detection in Learning Environments: A Review. Biosensors 2022, 12, 509.
  19. Richardson, D.C.; Griffin, N.K.; Zaki, L.; Stephenson, A.; Yan, J.; Curry, T.; Noble, R.; Hogan, J.; Skipper, J.I.; Devlin, J.T. Engagement in video and audio narratives: Contrasting self-report and physiological measures. Sci. Rep. 2020, 10, 11298.
  20. Ayres, P.; Lee, J.Y.; Paas, F.; van Merriënboer, J.J.G. The Validity of Physiological Measures to Identify Differences in Intrinsic Cognitive Load. Front. Psychol. 2021, 12, 702538.
  21. Pham, T.; Lau, Z.J.; Chen, S.H.A.; Makowski, D. Heart Rate Variability in Psychology: A Review of HRV Indices and an Analysis Tutorial. Sensors 2021, 21, 3998.
  22. Jerčić, P.; Sennersten, C.; Lindley, C. Modeling cognitive load and physiological arousal through pupil diameter and heart rate. Multimed. Tools Appl. 2018, 79, 3145–3159.
  23. Orazi, D.C.; Nyilasy, G. Straight to the Heart Of Your Target Audience. J. Advert. Res. 2019, 59, 137–141.
  24. Shaffer, F.; Ginsberg, J.P. An Overview of Heart Rate Variability Metrics and Norms. Front. Public Health 2017, 5, 258.
  25. van der Wel, P.; van Steenbergen, H. Pupil dilation as an index of effort in cognitive control tasks: A review. Psychon. Bull. Rev. 2018, 25, 2005–2015.
  26. Hensel, J.; Blackwell, R.; Sternthal, B. Pupil dilation: What does it measure? J. Advert. Res. 1970, 10, 15–19.
  27. Raiturkar, P.; Kleinsmith, A.; Keil, A.; Banerjee, A.; Jain, E. Decoupling light reflex from pupillary dilation to measure emotional arousal in videos. In Proceedings of the ACM Symposium on Applied Perception, Anaheim, CA, USA, 22–23 July 2016.
  28. Ioannou, S.; Gallese, V.; Merla, A. Thermal infrared imaging in psychophysiology: Potentialities and limits. Psychophysiology 2014, 51, 951–963.
  29. Engert, V.; Merla, A.; Grant, J.A.; Cardone, D.; Tusche, A.; Singer, T. Exploring the Use of Thermal Infrared Imaging in Human Stress Research. PLoS ONE 2014, 9, e90782.
  30. Adachi, H.; Oiwa, K.; Nozawa, A. Drowsiness Level Modeling Based on Facial Skin Temperature Distribution Using a Convolutional Neural Network. IEEE Trans. Electr. Electron. Eng. 2019, 14, 870–876.
  31. Iwashita, Y.; Nagumo, K.; Oiwa, K.; Nozawa, A. Estimation of resting blood pressure using facial thermal images by separating acute stress variations. Artif. Life Robot. 2021, 26, 473–480.
  32. Chaturvedi, I.; Thapa, K.; Cavallari, S.; Cambria, E.; Welsch, R.E. Predicting video engagement using heterogeneous DeepWalk. Neurocomputing 2021, 465, 228–237.
  33. Bulathwela, S.; P’erez-Ortiz, M.; Lipani, A.; Yilmaz, E.; Shawe-Taylor, J. Predicting Engagement in Video Lectures. arXiv 2020, arXiv:2006.00592.
  34. Wu, S.; Rizoiu, M.A.; Xie, L. Beyond Views: Measuring and Predicting Engagement in Online Videos. In Proceedings of the International AAAI Conference on Web and Social Media, Palo Alto, CA, USA, 25–28 June 2018; pp. 434–443.
  35. Anderson, A.; Hsiao, T.; Metsis, V. Classification of Emotional Arousal During Multimedia Exposure. In Proceedings of the PETRA ’17, 10th International Conference on PErvasive Technologies Related to Assistive Environments, Island of Rhodes, Greece, 21–23 June 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 181–184.
  36. Strle, G.; Košir, A.; Burnik, U. Physiological Signals and Affect as Predictors of Advertising Engagement. Sensors 2023, 23, 6916, 1-22.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Computer Science, Interdisciplinary Applications
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Gregor Strle
,
Andrej Košir
,
Urban Burnik
View Times: 550
Revision: 1 time (View History)
Update Date: 10 Aug 2023
Table of Contents
    1000/1000
    Hot Most Recent
    Video Production Service
    Feedback