XR, Vision Impairments and Empathy: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Jessie Wu and Version 5 by Peter Tang.

Across the world, an increasing number of people experience vision impairments or blindness. For people with healthy eyes, it may be difficult to discern how the world looks to a person with impaired eyesight. Medical explanations in books and articles, descriptions from patients, and 2D images of impaired vision are often insufficient to communicate the negative effects of vision impairments and to truly make someone understand how the world looks through the eyes of a visually impaired person. Relatives, employers, and even the medical providers of those impacted by eye diseases may benefit from simulations of vision impairments, to increase understanding, sympathy, and empathy. Research on digital vision-impairment simulations has increased. With the rise of emerging technologies such as extended reality (XR), simulations can now be made more realistic and immersive than ever before. ThisA document provides a cocomprehensive overview of the state-of-the-art in simulating vision impairments is provided using a variety of hardware and software solutions that address different conditions and impairments. It also discusses the role of XR solutions in increasing sympathy and empathy for people with vision impairments.

  • extended reality
  • virtual reality
  • augmented reality
  • eye tracking

1. XR and Vision Impairments

Over the last few decades, there have been a number of research and industrial efforts to accurately simulate vision impairments. This earlier work fits broadly into three categories: physical goggles with special lenses meant to simulate particular conditions, overlays of videos and simulations viewed on 2D displays, and XR systems that present dynamic representations of the impairments. These systems usually targeted specific conditions and were often limited in their realism, immersiveness, and adjustability.
Much of the earliest work, dating to the 1940s, used physical goggles and other obstructions to artificially reduce visual capabilities, with the goal of helping people with normal sight better understand the impact of low vision [1][2]. Zimmerman’s Low Vision Simulation Kit [3] used goggles with exchangeable lenses to represent different conditions. Later, Zagar and Baggarly [4] also utilized goggles to help student pharmacists understand how patients with various ocular diseases and vision impairments might interact with medication. Specific goggles were created per condition, including glaucoma, cataracts, macular degeneration, diabetic retinopathy, and retinitis pigmentosa.
Over the last decade, these types of physical instruments have been used to study how different conditions impact a patient’s life. Wood et al. [5] used goggles to help understand the impact of cataracts and refractive blur on night driving, showing the heaviest degradation in capability caused by the former. Hwang et al. [6] built on this and used a Bangerter Occlusion Foil on plano lenses to simulate cataracts, showing how headlight-induced glare reduces pedestrian visibility for those with the condition.
With the introduction of more powerful computing and display technologies, researchers moved to developing a number of static and dynamic 2D simulations. Greenberg [7] recreated the Farnsworth–Munsell 100-hue test using computer monitors to assess color vision deficiency, with a particular focus on presenting images as they would be seen by dichromats. Later, Brettel et al. [8] and Viénot et al. [9] both presented simulations of dichromacy by reprojecting images using LMS color spaces for viewing on monitors. Expanding to additional conditions, Banks and Crindle [10] recreated visual effects of several ocular diseases (including glaucoma and AMD) by creating overlays and filters for 2D images viewed on a desktop display.
These technologies also help application and web developers to better accommodate impaired users. Simulators such as VisionSimulations.com [11], Goodman-Deane et al. [12], and Leventhal [13] allow developers to assess the accessibility of websites, with the latter visualizing visual snow, glare, cataracts, and floaters, among others. Over the last 15 years, a new generation of tools has appeared, taking advantage of the emergence of 3D game engines to create more dynamic and immersive simulations. Lewis et al. [14][15] presented a set of simulations built using post-processing effects rendered in Unreal Engine 3, allowing for the simulation of several conditions in explorable 3D environments. Experts found the simulations educational and useful, even given the limitations on the customizability of the visualizations.
In the last decade, XR technology has become more immersive, powerful, and affordable, running on commodity consumer-grade hardware, with a number of manufacturers integrating eye-tracking technology. This novel technology allows for simulations with the immersiveness of goggles and the fidelity and customizability of 3D desktop systems. Additionally, the introduction of eye tracking facilitates the creation of much more dynamic visualizations that respond to a user’s gaze. For many conditions (e.g., glaucoma), this is critical to properly representing their appearance.
Werfel et al. [16], Väyrynen et al. [17], and Zavlanou and Lanitis [18] all developed VR simulations of vision impairments. While immersive and representative of the conditions, they were limited in their ability to tune symptoms to virtually change the severity and characteristics of the conditions. Additionally, state-of-the-art hardware has allowed for much more realistic rendering. Jones and Ometto [19] utilized eye tracking to achieve near real-time rendering for their XR simulations of disability glare, blur, spatial distortions, and perceptual filling-in, as well as defects in color vision. Furthermore, in their later work [20], they simulated glaucoma with gaze-dependent region blur, generated using perimetric data of a real patient. In a similar vein, Zhang et al. presented an AR system that allowed for the simulation of peripheral vision loss [21], utilizing monochrome LCDs to create virtual blind spots overlaid on the user’s view of the real world.
Commercially available apps (e.g., the Novartis ViaOpta Simulator [22] and VisionSim [23] from the Braille Institute) simulate vision impairments on a smartphone screen but are, consequently, monoscopic, limited in field of view, and unable to properly represent gaze-dependent effects.
XREye [24] builds on all of these approaches by simulating complex eye diseases, such as AMD, cornea disease, and achromatopsia, comprising multiple symptoms in real time with eye tracking, making it possible to use these simulations in XR HWDs while minimizing the risk of VR sickness. Most of the aforementioned earlier simulations are often simplifications of the respective vision impairments they aim to simulate. In an attempt to create more realistic representations, Krösl et al. [24] collaborated closely with ophthalmologists and modeled their simulations on the expertise of the medical experts and the reports from patients. With these additional considerations, XREye achieves more sophisticated visualizations of complex eye disease patterns.

2. XR for Sympathy and Empathy

The immersive factor of XR and the ability to simulate different spatial contexts and viewpoints for users has increased research interest in XR as an empathy-enabling medium over the past decade [25]. In broad terms, empathy can be defined as the capacity of feeling another person’s emotions. It is deeper than sympathy, the capacity to feel with the other (i.e., not the same feeling of the other, but rather a feeling of sorrow and care for the emotion of others) [26]. Sympathy and empathy are interconnected, but it is the possibility of “seeing through the eyes of others” using XR HWDs that makes the technology especially interesting for researching empathetic responses. For example, VR has become a prominent medium for nonfiction storytelling, with many documentaries making viewers experience the position of the subject of the story, aiming for an empathetic connection [27].
Empathy is a key factor in understanding the lives and symptoms of people with disabilities or age-related diseases in order to provide appropriate treatment. Therefore, the impact of using XR to foster empathy in students of medicine, nursing, and health sciences has been a topic of research. Campbell et al. [28] reported statistically relevant changes in pre/post surveys and improvement trends in empathy levels for Alzheimer’s dementia patients in undergraduate baccalaureate nursing students. Adefila et al. [29] also reported a statistically significant improvement in compassion among students for dementia patients in a pilot study in a faculty of health and life sciences. The study also interviewed the participants three months after the experience and collected testimonials on their interactions with dementia patients. Insights gained from interviews suggest that a VR experience might change work practices in the long term. Oh et al. [30] discuss the potential limitations of immersive technologies in fostering empathy. In their investigation of how embodied perspective-taking may help mitigate ageism and intergenerational bias, they found a more significant effect for participants who embodied an elderly person in an immersive virtual environment compared to those who engaged in a traditional perspective-taking exercise via mental simulation when a threat was presented without intergroup contact. However, they could not find a substantial difference between the two methods in the participants’ behaviors when they were exposed to a concrete and experiential intergroup threat. Paananen et al. [25] presented a systematic literature survey of recent work on empathy in XR and pointed out that, while current research presents evidence of XR experiences generating empathetic responses in the users, this impact tends to be limited, and not much research has been carried out to design guidelines for applications that foster long-term empathy in users.
The work by Guarese et al. [31], which focused on assistive technology, presented an approach for sympathy and empathy evaluations based on the Ad Response Sympathy (ARS) and Ad Response Empathy (ARE) tests from Escalas and Stern [26]. Guarese et al. developed an AR device as an assistive technology for micro-guidance in simple object-placement tasks in a kitchen environment. They evaluated its effect on sympathy and empathy in an experiment with blindfolded individuals. To gather data, they conducted an online survey as a pre-test to collect sighted people’s self-perception of sympathy and empathy for individuals with vision impairments. They also conducted a post-test with another group of sighted people after testing the developed assistive technology solution. Additionally, they assessed blind and visually impaired people’s perception of sympathy and empathy from others and compared these assessments, along with data from the pre-test and post-test. The results showed a significant increase in sympathetic and empathetic responses from the sighted people who participated in the experiment.
In an attempt to allow designers to test the usability and accessibility of their websites and desktop applications, Choo et al. [32] presented Empath-D to enable Empathetic User Interface Design. Using a Samsung Gear VR with a Galaxy Note 4, their application was aimed at helping designers account for motor problems. They could simulate a set of conditions by utilizing impairment profiles, with the ability to represent interactions of a person with a certain impediment, as well as visual impairments. This was later extended by Kim et al. [33] to work in a virtual environment for the design of smartphone apps. Empath-D was more recently evaluated by Choo et al. [34] using an augmented virtuality impairment simulation of a mockup of the Instagram app. The goal was to investigate usability challenges for patients with cataracts when using mobile apps. While the intensity of each effect was adjustable at run-time, the included impairment simulations were mostly simplified versions of the respective eye diseases, without eye tracking. Even though this system was developed to facilitate empathetic user interface design, its effects on the empathy levels of users have not been evaluated.
Building on the sympathy and empathy questionnaire developed by Guarese et al. [31], Krösl et al. [35]. conducted a study to evaluate the potential of their vision-impairment simulation system, XREye, to increase sympathy and empathy and assess its educational value. In the user study, 56 educators and education students either tested the vision impairment simulations themselves, using a VR HWD or observed another user's XR view projected on a wall. The study showed an increase in sympathy and empathy after experiencing XREye, along with positive feedback regarding its educational value. The authors suggest that vision-impairment simulations, such as XREye, can be used for educational purposes to increase awareness for the challenges people with vision impairments face in their everyday lives.

References

  1. Apple, L.; Apple, M.; Blasch, D. The artificial reduction of visual cues as a means of preparing training programs for low vision clients. Low Vis. Abstr. 1976, 2, 4–6.
  2. Bozeman, L.A.; Chang, C.H.S. Central field loss in object-system and low-vision simulation. Bull. Spec. Educ. 2006, 31, 207–220.
  3. Zimmerman Low Vision Simulation Kit. 2013. Available online: https://www.lowvisionsimulationkit.com (accessed on 30 October 2020).
  4. Zagar, M.; Baggarly, S. Low vision simulator goggles in pharmacy education. Am. J. Pharm. Educ. 2010, 74, 83.
  5. Wood, J.; Chaparro, A.; Carberry, T.; Chu, B.S. Effect of simulated visual impairment on nighttime driving performance. Optom. Vis. Sci. 2010, 87, 379–386.
  6. Hwang, A.D.; Tuccar-Burak, M.; Goldstein, R.; Peli, E. Impact of oncoming headlight glare with cataracts: A pilot study. Front. Psychol. 2018, 9, 164.
  7. Meyer, G.W.; Greenberg, D.P. Color-defective vision and computer graphics displays. IEEE Comput. Graph. Appl. 1988, 8, 28–40.
  8. Brettel, H.; Viénot, F.; Mollon, J.D. Computerized simulation of color appearance for dichromats. JOSA A 1997, 14, 2647–2655.
  9. Viénot, F.; Brettel, H.; Mollon, J.D. Digital video colourmaps for checking the legibility of displays by dichromats. In Color Research & Application: Endorsed by Inter-Society Color Council, the Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, the Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1999; Volume 24, pp. 243–252.
  10. Banks, D.; McCrindle, R. Visual eye disease simulator. In Proceedings of the 7th ICDVRAT with ArtAbilitation, Maia, Portugal, 8–10 September 2008.
  11. Simulate Your Vision For Family, Friends, and Eye Doctors. Available online: https://visionsimulations.com (accessed on 30 October 2019).
  12. Goodman-Deane, J.; Langdon, P.M.; Clarkson, P.J.; Caldwell, N.H.; Sarhan, A.M. Equipping designers by simulating the effects of visual and hearing impairments. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, Tempe, AZ, USA, 15–17 October 2007; pp. 241–242.
  13. Aaron Leventhal. NoCoffee—Vision Simulator for Chrome. 2013. Available online: https://accessgarage.wordpress.com/ (accessed on 30 October 2019).
  14. Lewis, J.; Brown, D.; Cranton, W.; Mason, R. Simulating visual impairments using the Unreal Engine 3 game engine. In Proceedings of the Serious Games and Applications for Health, Braga, Portugal, 16–18 November 2011; IEEE: Piscataway Township, NJ, USA, 2011; pp. 1–8.
  15. Lewis, J.; Shires, L.; Brown, D. Development of a visual impairment simulator using the Microsoft XNA Framework. In Proceedings of the 9th International Conference on Disability, Virtual Reality & Associated Technologie, Laval, France, 10–12 September 2012.
  16. Werfel, F.; Wiche, R.; Feitsch, J.; Geiger, C. Empathizing audiovisual sense impairments: Interactive real-time illustration of diminished sense perception. In Proceedings of the 7th Augmented Human International Conference, Geneva, Switzerland, 25–27 February 2016; ACM: New York, NY, USA, 2016; p. 15.
  17. Väyrynen, J.; Colley, A.; Häkkilä, J. Head mounted display design tool for simulating visual disabilities. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia, Rovaniemi, Finland, 12–15 December 2016; ACM: New York, NY, USA, 2016; pp. 69–73.
  18. Zavlanou, C.; Lanitis, A. Virtual Reality-Based Simulation of Age-Related Visual Deficiencies: Implementation and Evaluation in the Design Process. In Proceedings of the International Conference on Human Interaction and Emerging Technologies, Nice, France, 22–24 August 2019; Springer: Berlin/Heidelberg, Germany, 2019; pp. 262–267.
  19. Jones, P.R.; Ometto, G. Degraded Reality: Using VR/AR to simulate visual impairments. In Proceedings of the 2018 IEEE Workshop on Aug. and Virt. Realities for Good, Reutlingen, Germany, 18 March 2018; pp. 1–4.
  20. Jones, P.R.; Somoskeöy, T.; Chow-Wing-Bom, H.; Crabb, D.P. Seeing other perspectives: Evaluating the use of virtual and augmented reality to simulate visual impairments (OpenVisSim). Npj Digit. Med. 2020, 3, 1–9.
  21. Zhang, Q.; Barbareschi, G.; Huang, Y.; Li, J.; Pai, Y.S.; Ward, J.; Kunze, K. Seeing our Blind Spots: Smart Glasses-based Simulation to Increase Design Students’ Awareness of Visual Impairment. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, Bend, OR, USA, 29 October– 2 November 2022; pp. 1–14.
  22. Novartis Pharma, A.G. ViaOpta Simulator. 2018. Available online: https://www.viaopta-apps.com/ViaOpta-Simulator.html (accessed on 19 September 2019).
  23. Braille Institute of America. VisionSim by Braille Institute. Available online: https://apps.apple.com/us/app/visionsim-by-braille-institute/id525114829 (accessed on 30 October 2019).
  24. Krösl, K.; Elvezio, C.; Hürbe, M.; Karst, S.; Feiner, S.; Wimmer, M. XREye: Simulating visual impairments in eye-tracked XR. In Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA, 22–26 March 2020; IEEE: Piscataway Township, NJ, USA, 2020; pp. 830–831.
  25. Paananen, V.; Kiarostami, M.S.; Lee, L.H.; Braud, T.; Hosio, S. From Digital Media to Empathic Reality: A Systematic Review of Empathy Research in Extended Reality Environments. arXiv 2022, arXiv:2203.01375.
  26. Escalas, J.E.; Stern, B.B. Sympathy and Empathy: Emotional Responses to Advertising Dramas. J. Consum. Res. 2003, 566.
  27. Bevan, C.; Green, D.P.; Farmer, H.; Rose, M.; Cater, K.; Fraser, D.S.; Brown, H. Behind the Curtain of the “Ultimate Empathy Machine": On the Composition of Virtual Reality Nonfiction Experiences; Association for Computing Machinery: New York, NY, USA, 2019.
  28. Campbell, D.; Lugger, S.; Sigler, G.S.; Turkelson, C. Increasing awareness, sensitivity, and empathy for Alzheimer’s dementia patients using simulation. Nurse Educ. Today 2021, 98, 104764.
  29. Adefila, A.; Graham, S.; Clouder, L.; Bluteau, P.; Ball, S. myShoes—The future of experiential dementia training? J. Ment. Health Train. Educ. Pract. 2016, 11, 91–101.
  30. Oh, S.Y.; Bailenson, J.; Weisz, E.; Zaki, J. Virtually old: Embodied perspective taking and the reduction of ageism under threat. Comput. Hum. Behav. 2016, 60, 398–410.
  31. Guarese, R.; Pretty, E.; Fayek, H.; Zambetta, F.; van Schyndel, R. Evoking empathy with visually impaired people through an augmented reality embodiment experience. In Proceedings of the 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), Shanghai, China, 25–29 March 2023.
  32. Choo, K.T.W.; Balan, R.K.; Wee, T.K.; Chauhan, J.; Misra, A.; Lee, Y. Empath-d: Empathetic design for accessibility. In Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications, Sonoma, CA, USA, 21–22 February 2017; pp. 55–60.
  33. Kim, W.; Choo, K.T.W.; Lee, Y.; Misra, A.; Balan, R.K. Empath-d: Vr-based empathetic app design for accessibility. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, Munich, Germany, 10–15 June 2018; pp. 123–135.
  34. Choo, K.T.W.; Balan, R.K.; Lee, Y. Examining Augmented Virtuality Impairment Simulation for Mobile App Accessibility Design. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019; pp. 1–11.
  35. Katharina Krösl; Marina Lima Medeiros; Marlene Huber; Steven Feiner; Carmine Elvezio Exploring the Educational Value and Impact of Vision-Impairment Simulations on Sympathy and Empathy with XREye. Multimodal Technol. Interact. 2023, 7, 70.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback