Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1
Fotis Liarokapis
 -- 1791 2024-01-19 15:05:08 |
2 layout & references
Sirius Huang
 Meta information modification 1791 2024-01-22 02:10:20 |

Video Upload Options

Do you have a full video?
Send video materials Upload full video

Confirm

Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Apostolou, K.; Škola, F.; Liarokapis, F. Vibrotactile Feedback in Virtual Reality. Encyclopedia. Available online: https://encyclopedia.pub/entry/54125 (accessed on 08 May 2024).
Apostolou K, Škola F, Liarokapis F. Vibrotactile Feedback in Virtual Reality. Encyclopedia. Available at: https://encyclopedia.pub/entry/54125. Accessed May 08, 2024.
Apostolou, Kalliopi, Filip Škola, Fotis Liarokapis. "Vibrotactile Feedback in Virtual Reality" Encyclopedia, https://encyclopedia.pub/entry/54125 (accessed May 08, 2024).
Apostolou, K., Škola, F., & Liarokapis, F. (2024, January 19). Vibrotactile Feedback in Virtual Reality. In Encyclopedia. https://encyclopedia.pub/entry/54125
Apostolou, Kalliopi, et al. "Vibrotactile Feedback in Virtual Reality." Encyclopedia. Web. 19 January, 2024.
Vibrotactile Feedback in Virtual Reality
Edit
This entry is adapted from the peer-reviewed paper 10.3390/mti7120114

While substantial progress in computer graphics and sound rendering has resulted in highly realistic visual and auditory experiences in virtual reality (VR), achieving genuine immersion, interactivity, and the stimulation of imagination necessitates the integration of realistic tactile experiences, often facilitated through haptic feedback. The incorporation of vibrotactile feedback in VR allows users to fully engage their sense of touch, enabling them to explore, grasp, and manipulate virtual objects as if they were interacting with them in the physical world.

vibrotactile funneling illusion virtual reality haptic devices auditory feedback localization task multisensory environments avatar hands

1. Introduction

Creating a seamless integration of various sensory inputs poses a notable challenge in the domain of virtual reality (VR) [1]. While substantial progress in computer graphics and sound rendering over the past five decades has resulted in highly realistic visual and auditory experiences in VR [1][2], achieving genuine immersion, interactivity, and the stimulation of imagination [3] necessitates the integration of realistic tactile experiences, often facilitated through haptic feedback [4]. Haptic feedback encompasses diverse tactile stimulation methods, including vibrations, forces, and motions, all aimed at conveying information and enriching user experiences [5]. This text's specific focus lies in vibrotactile feedback, a haptic feedback form utilizing controlled vibrations to replicate sensations on the skin [6]. This technology seamlessly integrates into various devices, such as touchscreens, game controllers, and wearables, offering users realistic and immersive sensations that closely mimic physical attributes like texture, shape, weight, temperature, or resistance [7][8][9]. Consequently, the incorporation of vibrotactile feedback in VR allows users to fully engage their sense of touch, enabling them to explore, grasp, and manipulate virtual objects as if they were interacting with them in the physical world [9][10][11].
In the domain of haptic perception, a noteworthy area of research explores the ‘funneling effect,’ initially observed by v. Békésy [12]. The ‘tactile funneling illusion’ is characterized by the emergence of phantom tactile sensations through the application of vibrotactile feedback [13]. In this context, ‘phantom’ sensations refer to illusory tactile experiences, often perceived as occurring between distant points of skin stimulation [14][15][16][17]. This research aims to better understand and manipulate the tactile perception of users by exploring how vibrotactile feedback can create these intriguing illusory sensations. The tactile funneling technique has regained significant attention, especially in the contexts of virtual and augmented reality [8][13][18][19][20][21][22][23][24]. It has the potential to expand users’ perception of touch to virtual objects they interact with. For instance, in VR environments, when handheld controllers deliver vibrotactile sensations, the funneling illusion creates a compelling sensation for users. It makes them feel as if they are genuinely touching virtual objects or sensing the impact on a virtual object caused by interactions with other virtual objects, simulating forces [13][20][21][22]. This unique phenomenon significantly elevates the immersive quality of VR experiences by effectively bridging the gap between the digital and physical worlds.
The robustness of the funneling effect has been evident across various skin locations, including the palms, fingers, arms, and even the forehead [18]. However, despite its demonstrated reliability, prior research on the tactile funneling illusion has exhibited limitations in its scope. The majority of studies have primarily focused on a limited set of skin locations, typically involving just five intended illusory points [8][13][19][20][21][22][23]. In a few instances, researchers have expanded their investigations to include up to seven points [18][25]. Nevertheless, these studies have often overlooked multisensory environments, particularly neglecting audio stimulation. As a result, a notable gap remains in our understanding of how the tactile funneling illusion operates in multisensory contexts, where the integration of various sensory modalities has the potential to enhance its effectiveness. Furthermore, our current knowledge gap extends to understanding how these factors may interact and potentially reduce the perceived workload, a crucial aspect of enhancing user experiences in such settings [26].

2. Assessing Tactile Funneling in VR: An Objective Approach

The funneling effect, a phenomenon achieved by delivering tactile stimuli to various areas of the skin in a specific pattern, intensifies tactile sensations and directs them toward a central point on the skin [12][15][16][17]. For example, equal intensity delivered to both the right and left hands creates a phantom sensation centered between them. However, by varying the intensity, such as increasing it in the left hand while keeping it lower in the right hand, the perception of touch appears to shift towards the left side, with the reverse occurring when the intensity is higher on the right side. This phenomenon demonstrates how we can manipulate the perception of touch by adjusting the intensity of vibrotactile feedback across different body areas, even including the forehead [18].
In previous research, participants’ descriptions of their experiences with phantom haptic events in VR settings have often been subjective. Many studies have relied on subjective questionnaires, administered in either virtual scenes or physical setups, to assess participants’ perceptions, often through Likert scale responses [8][13][20][21][23]. These questionnaires included inquiries such as, ‘I could locate where the events seemed to occur on the stick’. Alternatively, some studies adopted a raycasting method [19][22], requiring participants to utilize a foot pedal to accurately pinpoint the location where they sensed this phantom tactile sensation. These assessment methods introduced notable delays between experiencing the illusion and reporting it, potentially affecting response accuracy and our understanding of the illusion’s intricacies.
Another study [8] emphasized the significance of incorporating bimodal feedback (visual–haptic) to enhance both presence and haptic perception in VR. They conducted a study to investigate the impact of different feedback modalities (visual only, haptic only, and visual plus haptic) on users’ sense of presence. Surprisingly, their findings challenged the common assumption that visual feedback alone offers a better experience than haptic feedback alone, ultimately supporting the role of bimodal feedback (visual–haptic) in enhancing presence. However, none of the previous studies have investigated the potential of trimodal stimulation in VR experiments to further enhance haptic perception and presence when utilizing the tactile funneling technique.
Research on the funneling effect also extends to the design of cost-effective virtual haptic systems. For instance, Wang’s study [27] has highlighted the customization of haptic feedback in real time, considering events in the virtual environment. The research prioritizes key elements such as haptic feedback in collisions and responsive force algorithms, fostering engaging virtual experiences. Importantly, Wang’s study introduces a personalized approach, laying the groundwork for diverse haptic applications.

3. Multimodal Stimulation and VR: Performance Insights

Recent research investigated the influence of multimodal stimulation and perceptual load on human performance, workload, and presence in a VR target detection task [26]. To investigate these aspects comprehensively, the study employed a range of methodologies, including questionnaires, behavioral assessments, and neurophysiological measures such as Electroencephalography (EEG) and Galvanic Skin Response (GSR). EEG assessed task difficulty, while GSR objectively measured body illusions, as observed in earlier studies [28][29]. The study presented participants with various multimodal stimuli, including auditory and vibrotactile cues, either in isolation or in combination with visual targets. The research found that combining visual, auditory, and vibrotactile cues (trimodal) or using visual–vibrotactile stimulation (bimodal) significantly improved performance in high-perceptual-load VR tasks compared to visual stimulation alone. EEG data analysis showed that multisensory integration, especially with visual–audio–tactile and visual–audio cues, enhanced stimulus processing speed (i.e., faster detection of stimuli). However, this effect was not observed with visual–tactile stimulation. Both trimodal and bimodal stimulation reduced EEG-based workload, but only trimodal reduced subjective workload according to the NASA Task Load Index (NASA-TLX) (i.e., perceived workload). Interestingly, high-demand VR tasks increased the sense of presence, with trimodal stimulation being more effective than bimodal or unimodal. Additionally, skin conductance levels increased with task demands in VR, aligning with real-world task observations.
In a related systematic review by Melo et al. [1], which examined 105 studies on human senses in virtual environments, it was shown that 84.8% of these studies reported positive outcomes for multisensory VR experiences when compared to bimodal or unimodal experiences. Haptic feedback emerged as a dominant feature, featured in 86.6% of the studies, highlighting its crucial role in enhancing sensory immersion within virtual settings. These findings collectively help us understand how different types of stimulation and task complexity in VR impact performance, workload perception, and the sense of presence, providing insights for improvement.

4. Avatar Embodiment in VR: Effects on Tactile Perception

Avatar embodiment is a multifaceted concept encompassing ownership perception, appearance realism, sensory integration, and responsive interactions [30]. Previous work emphasizes the importance of full-body avatars in VR, particularly in enhancing vibrotactile [22] and visual–vibrotactile stimulation [31][32]. The presence of a virtual body significantly improves users’ spatialization of touch, crucial in scenarios involving ambiguous haptic feedback.
Interestingly, participants’ objective and subjective reports of touch illusion sometimes differ, suggesting that subjective measures of immersion may not always align with users’ internal cognitive states [22]. Objective measures like haptic localization tasks are proposed to gain deeper insights. Additionally, embodying different avatars or adjusting virtual body characteristics (e.g., size, gender, or race) can lead to shifts in haptic perception [30].
Hence, integrating avatar bodies or avatar body parts into VR environments offers a unique opportunity to enhance avatar embodiment. Matching skin tones further strengthens the connection between users and their avatars, improving the VR experience [33].

5. The Neural Mechanisms behind Tactile Illusions: Insights from VR

Vibrotactile stimulation has gained popularity in recent years for enhancing human–computer interaction (HCI), particularly in VR. This often involves using vibrators in game controllers, but more advanced tactile grids with multiple vibrators pose cost and usability challenges. To address this, tactile illusions have emerged as a promising solution, creating ‘phantom’ vibrotactile sensations seemingly originating from non-contact points [13][20]. One technique, saltation, induces pseudo-tactile sensations between two body locations through timed stimuli [34], while funneling generates similar sensations by simultaneously stimulating two skin locations with varying amplitudes [16].
Despite their significance for effective use in human–computer interaction, there has been limited research into the physiological and neural mechanisms underlying tactile illusions. To address this gap, Lee et al. [21] conducted a study focusing on the brain activation patterns associated with the tactile funneling illusion—a phenomenon where tactile sensations are perceived as originating from external locations without any physical stimulator present. Using EEG to measure brain waves, the researchers compared illusory tactile sensations generated by the funneling technique to non-illusory sensations. Surprisingly, their results revealed that both non-illusory and illusory sensations involved the parietal lobe despite the absence of a corresponding body map area for an external object. Furthermore, illusory sensations exhibited additional processing, resulting in delays in event-related potential (ERP), and involvement of the limbic lobe. These findings suggest a possible connection to memory and recognition tasks.
In summary, this research reveals the complexities of tactile illusions and their neural basis. It suggests that humans may have limitations in recognizing multiple illusory sensations due to the constraints of working memory. Additionally, the study shows that the speed of brain responses (ERP latency) can affect how well these illusions work, especially when using different-sized devices or virtual objects in VR.

References

  1. Melo, M.; Goncalves, G.; Monteiro, P.; Coelho, H.; Vasconcelos-Raposo, J.; Bessa, M. Do Multisensory Stimuli Benefit the Virtual Reality Experience? A Systematic Review. IEEE Trans. Vis. Comput. Graph. 2022, 28, 1428–1442.
  2. Apostolou, K.; Liarokapis, F. A Systematic Review: The Role of Multisensory Feedback in Virtual Reality. In Proceedings of the 2022 IEEE 2nd International Conference on Intelligent Reality (ICIR), Virtual, 14–16 December 2022; pp. 39–42.
  3. Sutherland, I.E. The ultimate display. In Proceedings of the IFIP Congress, New York, NY, USA, 24–29 May 1965; Volume 2, pp. 506–508.
  4. Wang, D.; Guo, Y.; Liu, S.; Zhang, Y.; Xu, W.; Xiao, J. Haptic display for virtual reality: Progress and challenges. Virtual Real. Intell. Hardw. 2019, 1, 136–162.
  5. Kreimeier, J.; Hammer, S.; Friedmann, D.; Karg, P.; Bühner, C.; Bankel, L.; Götzelmann, T. Evaluation of Different Types of Haptic Feedback Influencing the Task-Based Presence and Performance in Virtual Reality. In Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, New York, NY, USA, 22–25 October 2019; PETRA ’19. pp. 289–298.
  6. Lindeman, R.W.; Page, R.; Yanagida, Y.; Sibert, J.L. Towards Full-Body Haptic Feedback: The Design and Deployment of a Spatialized Vibrotactile Feedback System. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, New York, NY, USA, 10–12 November 2004; VRST ’04. pp. 146–149.
  7. Cabaret, P.A.; Howard, T.; Gicquel, G.; Pacchierotti, C.; Babel, M.; Marchal, M. Does Multi-Actuator Vibrotactile Feedback Within Tangible Objects Enrich VR Manipulation? IEEE Trans. Vis. Comput. Graph. 2023, 1–13.
  8. Gibbs, J.K.; Gillies, M.; Pan, X. A comparison of the effects of haptic and visual feedback on presence in virtual reality. Int. J. Hum.-Comput. Stud. 2022, 157, 102717.
  9. Moosavi, M.S.; Raimbaud, P.; Guillet, C.; Plouzeau, J.; Merienne, F. Weight perception analysis using pseudo-haptic feedback based on physical work evaluation. Front. Virtual Real. 2023, 4, 973083.
  10. Ernst, M.O.; Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002, 415, 429–433.
  11. Ramsamy, P.; Haffegee, A.; Jamieson, R.; Alexandrov, V. Using haptics to improve immersion in virtual environments. In Proceedings of the Computational Science–ICCS 2006: 6th International Conference, Reading, UK, 28–31 May 2006; Proceedings, Part II 6. pp. 603–609.
  12. Békésy, G.V. Funneling in the Nervous System and its Role in Loudness and Sensation Intensity on the Skin. J. Acoust. Soc. Am. 1958, 30, 399–412.
  13. Lee, J.; Kim, Y.; Kim, G. Funneling and saltation effects for tactile interaction with virtual objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012; pp. 3141–3148.
  14. Park, G.; Choi, S. Tactile Information Transmission by 2D Stationary Phantom Sensations. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21–26 April 2018; pp. 1–12.
  15. Chen, L.M.; Friedman, R.M.; Roe, A.W. Optical Imaging of a Tactile Illusion in Area 3b of the Primary Somatosensory Cortex. Science 2003, 302, 881–885.
  16. Alles, D. Information Transmission by Phantom Sensations. IEEE Trans. Man Mach. Syst. 1970, 11, 85–91.
  17. Hayward, V. A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res. Bull. 2008, 75, 742–752.
  18. Kaul, O.B.; Rohs, M.; Simon, B.; Demir, K.C.; Ferry, K. Vibrotactile Funneling Illusion and Localization Performance on the Head. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–13.
  19. Berger, C.C.; Gonzalez-Franco, M. Expanding the sense of touch outside the body. In Proceedings of the 15th ACM Symposium on Applied Perception, Vancouver, BC, Canada, 10–11 August 2018; pp. 1–9.
  20. Lee, J.; Kim, Y.; Kim, G.J. Effects of Visual Feedback on Out-of-Body Illusory Tactile Sensation When Interacting with Augmented Virtual Objects. IEEE Trans. Hum.-Mach. Syst. 2017, 47, 101–112.
  21. Lee, H.; Lee, J.; Kim, C.; Kim, G.; Kim, E.S.; Whang, M. Brain Process for Perception of the “Out of the Body” Tactile Illusion for Virtual Object Interaction. Sensors 2015, 15, 7913–7932.
  22. Gonzalez-Franco, M.; Berger, C.C. Avatar Embodiment Enhances Haptic Confidence on the Out-of-Body Touch Illusion. IEEE Trans. Haptics 2019, 12, 319–326.
  23. Berger, C.C.; Gonzalez-Franco, M.; Ofek, E.; Hinckley, K. The uncanny valley of haptics. Sci. Robot. 2018, 3, eaar7010.
  24. Apostolou, K.; Charalambous, M.; Makri, S.; Charalambous, P.; Liarokapis, F. The Funneling Effect: A prototype implementation of an illusory sense of touch in virtual reality. In Proceedings of the 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Shanghai, China, 25–29 March 2023; pp. 338–345.
  25. Kerdegari, H.; Kim, Y.; Stafford, T.; Prescott, T.J. Centralizing bias and the vibrotactile funneling illusion on the forehead. In Proceedings of the Haptics: Neuroscience, Devices, Modeling, and Applications: 9th International Conference, EuroHaptics 2014, Versailles, France, 24–26 June 2014; Proceedings, Part II 9. pp. 55–62.
  26. Marucci, M.; Di Flumeri, G.; Borghini, G.; Sciaraffa, N.; Scandola, M.; Pavone, E.F.; Babiloni, F.; Betti, V.; Aricò, P. The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence. Sci. Rep. 2021, 11, 4831.
  27. Wang, F.; Qian, Z.; Lin, Y.; Zhang, W. Design and Rapid Construction of a Cost-Effective Virtual Haptic Device. IEEE/ASME Trans. Mechatronics 2020, 26, 66–77.
  28. Gonzalez-Franco, M.; Perez-Marcos, D.; Spanlang, B.; Slater, M. The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment. In Proceedings of the 2010 IEEE Virtual Reality Conference (VR), Waltham, MA, USA, 25–29 March 2010; pp. 111–114.
  29. Slater, M.; Spanlang, B.; Sanchez-Vives, M.V.; Blanke, O. First person experience of body transfer in virtual reality. PLoS ONE 2010, 5, e10564.
  30. Gonzalez Franco, M.; Peck, T.C. Avatar Embodiment. Towards a Standardized Questionnaire. Front. Robot. AI 2018, 5, 74.
  31. Eagleman, D.M.; Sejnowski, T.J. Motion integration and postdiction in visual awareness. Science 2000, 287, 2036–2038.
  32. Maselli, A.; Kilteni, K.; López-Moliner, J.; Slater, M. The sense of body ownership relaxes temporal constraints for multisensory integration. Sci. Rep. 2016, 6, 30628.
  33. Maselli, A.; Slater, M. The building blocks of the full body ownership illusion. Front. Hum. Neurosci. 2013, 7, 83.
  34. Miyazaki, M.; Hirashima, M.; Nozaki, D. The “cutaneous rabbit” hopping out of the body. J. Neurosci. 2010, 30, 1856–1860.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Others
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Kalliopi Apostolou
,
Filip Škola
,
Fotis Liarokapis
View Times: 500
Revisions: 2 times (View History)
Update Date: 22 Jan 2024
Table of Contents
    1000/1000
    Hot Most Recent
    Feedback