Augmented and Virtual Reality in Dermatology: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Subjects: Dermatology
Contributor:
Mathias Bonmarin

Augmented reality (AR) is a component of virtual reality (VR) or virtual environments (VE) where the user is completely immersed into a synthetic environment. While in VR, the user does not see the real world. Instead, AR allows us to superimpose virtual objects onto real ones. The number of AR/VR applications has undeniably exploded over the last decades. While both techniques have been extensively used in various medical fields, such as in plastic surgery, little investigation has been performed so far in dermatology.

  • virtual reality
  • dermatology

1. Education

Improving dermatologic teaching is a challenge, as visual memory is difficult to maintain without constant exposure. As a result, it has been reported that the diagnostic accuracy of common skin diseases is poor by primary care physicians [1]. The use of 3D silicon-made phantoms greatly improves diagnostics skills and knowledge retention in comparison to standard 2D images [2]. Therefore, it is expected that AR should exhibit similar effects [3]. Nonetheless, recent studies have demonstrated ambiguous results with minor differences in knowledge gain and slightly better knowledge retention for the group using AR-based tools in comparison to textbooks [4]. This outcome could potentially be improved by coupling AR with haptic feedback, as is the case in most surgical training tools.
A potential application that could prove to be beneficial in training the next generation of medical experts would include realistic-looking 3D models displaying skin diseases and irregularities (see Figure 1). These models would contain high resolution clinical textures that are anonymized on newly generated 3D models with anatomical proportions, all of which could be run on smartphones/tablets or VR goggles. The environmental conditions (like lighting conditions) could be adapted for a more realistic experience. In addition, such models could be expanded by using synthetic clinical images, to reduce data scarcity and poor training regarding rare dermatological diseases and certain Fitzpatrick skin types. For example, incidence of melanoma is relatively rare among black populations, where survival rates lag behind the rates for non-Hispanic white populations in the US [5]. Such issues have recently been highlighted in the press [6]. 
Dermato 02 00001 g001 550
Figure 1. A 3D human, full body (middle) and with closeup to chest (left) and back (right). Photos are screenshots from a smartphone application.
Aside from health professionals, some studies have investigated AR/VR as an educational tool for patients or the public to increase their awareness of skin disease prevention. The group Elke Hacker in Australia has developed a virtual reality (VR) game containing preventative skin cancer messages for youth and young adults [7]. This was based on the fact that despite successful public campaigns in the 1980’s, high rates of sunburn continue to be reported among the young population.
In another application of VR in dermatology, we created the VR app Virtual Derm, which places students in a virtual practice for dermatology [8]. The app runs on standard Android smartphones, making it accessible to most students. A total of 100 patient vignettes can be solved by students, making it a unique learning opportunity (see Figure 2).
Dermato 02 00001 g002 550
Figure 2. Extracted pictures from the Virtual Derm app [8]. Dermatitis herpetiformis Duhring on the forearm of male adult homunculus (top). Melanoma on left flank of female homunculus (bottom).

2. Dermatologic Surgery

Dermatologic surgery is the practice of dermatology that specializes in surgical procedures to repair or improve the function and appearance of the skin [9]. While plastic surgery greatly benefited from AR/VR technology [10], few applications have been reported in dermatologic surgery. Aside from surgical procedure training [11], AR and VR have been implemented in Mohs surgery to reduce patient anxiety [12] or as a surgical assistance tool [13].
For patients, VR has been used to reduce pre- and intraoperative anxiety. The tools employed in this study were unspecific VR sequences which could be used to reduce anxiety under any circumstance. It has been shown that almost all anxiety-related parameters could be improved in the context of surgery [12]. More specifically, VR could be used as tool for patient education, as many patients do not have a clear understanding what a surgical procedure entails, such as using a flap for the reconstruction of a surgical defect. However, this has not been investigated and may also induce anxiety in some patients where too much detail about a surgical procedure is given.
For physicians, VR has been used as a training tool in many surgical specialties and has great potential in dermatologic surgery. The more realistic appearance of a VR image, combined with haptic sensations, can be used to train dermatologic surgical interventions in the same way as in other surgical fields, notably plastic surgery. AR has been successfully used by one group to improve the communication between surgeons and pathologists in cases of Mohs surgery, where the pathology slides were not necessarily read by the surgeon themselves [13]. In addition, AR has great potential for other aspects of Mohs surgery and dermatologic surgery. After acquiring the theoretical background and training with artificial models, a surgical procedure is best learnt by performing the procedure on a real patient. In most training programs, this is performed under the direct supervision of a senior faculty member who is present at the operating table. However, due to shortcomings in staff resources and planning, this is not always possible and junior surgeons sometimes find themselves performing an operation that is above their level of confidence alone. AR could allow a senior faculty member to follow an operation remotely and give very precise and practical instructions to the junior surgeon at the table, with AR images superimposed on the surgical site to guide the surgery. In larger institutions, this would also allow senior staff resources to be used more efficiently, as one experienced surgeon could follow several surgeries at the same time.

3. Diagnostics

Artificial intelligence (AI) has been productively utilized for several decades in dermatology, particularly for skin cancer detection. It has been reported that convolutional neural networks (CNNs) achieve dermatologist-level classification of cutaneous lesions using both dermoscopic and non-dermoscopic images [14]. AI has proven very useful when combined with VR reconstruction of the body surface. In a prominent example, the Canfield® Vectra WB360 device combines 2D photographs of 46 stereo-vision pods, resulting in a full-body VR reconstruction that can be turned and manipulated at will. The machine further analyzes all photographs with ML-based segmentation and classification, looking for naevi and other cutaneous lesions. All naevi can be analyzed by artificial intelligence for their probability of being malignant. The information on these lesions is saved, with the position of each lesion on the body map, and upon repeated patient use of the Vectra device, changes are tracked over time. By utilizing this technology, newly developing melanomas are actively being found.
AI-based smartphone applications, or “apps”, are being successfully developed as practical tools to improve diagnostics [15][16][17][18], despite some limitations in terms of accuracy for melanoma screening [17].
The use of AR/VR in dermatology could strongly benefit from haptic feedback. Haptic feedback, or “haptics”, refers to manual interaction with the environment, and could be used to relay force and tactile information to the user [19]. Indeed, sensory information such as skin temperature, roughness or stiffness would provide valuable additional information to support diagnostic evaluation [20]. In this direction, Kim and co-workers developed a method to convert a single image to a 3D haptic surface and render the generated haptic surface in real-time [21][22].

This entry is adapted from the peer-reviewed paper 10.3390/dermato2010001

References

  1. Federman, D.G.; Kirsner, R.S. The abilities of primary care physicians in dermatology: Implications for quality of care. Am. J. Manag. Care 1997, 3, 1487–1492.
  2. Garg, A.; Haley, H.-L.; Hatem, D. Modern Moulage Evaluating the Use of 3-Dimensional Prosthetic Mimics in a Dermatology Teaching Program for Second-Year Medical Students. Arch. Dermatol. 2010, 146, 143–146.
  3. Aldridge, R.B.; Li, X.A.; Ballerini, L.; Fisher, R.B.; Rees, J.L. Teaching Dermatology Using 3-Dimensional Virtual Reality. Arch. Dermatol. 2010, 146, 1184–1185.
  4. Noll, C.; von Jan, U.; Raap, U.; Albrecht, U.-V. Mobile Augmented Reality as a Feature for Self-Oriented, Blended Learning in Medicine: Randomized Controlled Trial. JMIR mHealth uHealth 2017, 5, e7943.
  5. Culp, M.B.; Lunsford, N.B. Melanoma among Non-Hispanic Black Americans. Prev. Chronic Dis. 2019, 16.
  6. Caryn Rabin, R. Dermatology’s Skin Color Problem. The New York Times, 8 September 2020; 1.
  7. Horsham, C.; Dutton-Regester, K.; Antrobus, J.; Goldston, A.; Price, H.; Ford, H.; Hacker, E. A Virtual Reality Game to Change Sun Protection Behavior and Prevent Cancer: User-Centered Design Approach. JMIR Serious Games 2021, 9, e24652.
  8. Virtual Derm Is a Health Education app, Meant to Provide a Training Platform to Help Medical Students and Dermatologists Training Their Observational, Diagnostic and Treatment/Care Skills for a Better Patient Care in Dermatology. Version 2.0 Available for Download on Google Play. Available online: https://play.google.com/store/apps/details?id=com.HumanGames.VirtualDerm2&hl=en&gl=US (accessed on 20 December 2021).
  9. Hale, E. Handbook of Dermatologic Surgery; Springer: New York, NY, USA, 2014.
  10. Khor, W.S.; Baker, B.; Amin, K.; Chan, A.; Patel, K.; Wong, J. Augmented and virtual reality in surgery-the digital surgical environment: Applications, limitations and legal pitfalls. Ann. Transl. Med. 2016, 4.
  11. Berg, D.; Raugi, G.; Gladstone, H.; Berkley, J.; Weghorst, S.; Ganter, M.; Turkiyyah, G. Virtual reality simulators for dermatologic surgery: Measuring their validity as a teaching tool. Dermatol. Surg. 2001, 27, 370–374.
  12. Higgins, S.; Feinstein, S.; Hawkins, M.; Cockburn, M.; Wysong, A. Virtual Reality to Improve the Experience of the Mohs Patient-A Prospective Interventional Study. Dermatol. Surg. 2019, 45, 1009–1018.
  13. Rodriguez-Jimenez, P.; Ruiz-Rodriguez, R. Augmented reality in Mohs micrographic surgery. Int. J. Dermatol. 2020, 59, E22–E23.
  14. Young, A.T.; Xiong, M.L.; Pfau, J.; Keiser, M.J.; Wei, M.L. Artificial Intelligence in Dermatology: A Primer. J. Investig. Dermatol. 2020, 140, 1504–1512.
  15. Francese, R.; Frasca, M.; Risi, M.; Tortora, G. A mobile augmented reality application for supporting real-time skin lesion analysis based on deep learning. J. Real-Time Image Process. 2021, 18, 1247–1259.
  16. Freeman, K.; Dinnes, J.; Chuchu, N.; Takwoingi, Y.; Bayliss, S.E.; Matin, R.N.; Jain, A.; Walter, F.M.; Williams, H.C.; Deeks, J.J. Algorithm based smartphone apps to assess risk of skin cancer in adults: Systematic review of diagnostic accuracy studies. Bmj-Br. Med. J. 2020, 368.
  17. Sun, M.D.; Kentley, J.; Mehta, P.; Duzsa, S.; Halpern, A.C.; Rotemberg, V. Accuracy of commercially available smartphone applications for the detection of melanoma. Br. J. Dermatol. 2021.
  18. Chuchu, N.; Takwoingi, Y.; Dinnes, J.; Matin, R.N.; Bassett, O.; Fmoreau, J.; Bayliss, S.E.; Davenport, C.; Godfrey, K.; O’Connell, S.; et al. Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma. Cochrane Database Syst. Rev. 2018, 12.
  19. Srinivasan, M.A.; Basdogan, C. Haptics in virtual environments: Taxonomy, research status, and challenges. Comput. Graph. 1997, 21, 393–404.
  20. Waldron, K.J.; Enedah, C.; Gladstone, H. Stiffness and texture perception for teledermatology. Stud. Health Technol. Inform. 2005, 111, 579–585.
  21. Kim, K.; Lee, S. Perception-based 3D tactile rendering from a single image for human skin examinations by dynamic touch. Ski. Res. Technol. 2015, 21, 164–174.
  22. Kim, K. Roughness based perceptual analysis towards digital skin imaging system with haptic feedback. Ski. Res. Technol. 2016, 22, 334–340.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
Video Production Service
Feedback