Augmented Reality in Neurosurgery: History
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor: , , ,

Augmented reality (AR) involves the overlay of computer-generated images onto the user’s real-world visual field to modify or enhance the user’s visual experience. With respect to neurosurgery, AR integrates preoperative and intraoperative imaging data to create an enriched surgical experience that has been shown to improve surgical planning, refine neuronavigation, and reduce operation time. In addition, AR has the potential to serve as a valuable training tool for neurosurgeons in a way that minimizes patient risk while facilitating comprehensive training opportunities.

  • augmented reality
  • machine learning
  • neurosurgery
  • cranial neurosurgery

1. Introduction

AR technology involves the superimposition of computer-generated images onto the user’s real-world visual field to modify or enhance the user’s visual experience. In medicine, AR has been utilized in various surgical subspecialties, including ophthalmology, general surgery, orthopedic surgery, urology, and oral and maxillofacial surgery for its ability to integrate computer-generated images with intraoperative visualization of the surgical field [1]. The field of neurosurgery in particular has been at the forefront of image-guided surgery since the early 1990s, as surgeons must demonstrate high intraoperative accuracy and precision to navigate microscopic targets in a way that maximizes patient outcomes while also preserving neurologic functioning. Conventional neuronavigation and neuroimaging technologies involve the use of two-dimensional images to guide surgical planning and neuronavigation. However, integration of preoperative and/or intraoperative magnetic resonance imaging (MRI), computerized tomography (CT), angiography, and tractography images into the real-world surgical environment provides neurosurgeons with an enriched, three-dimensional (3D), semi-immersive experience designed to improve surgical planning, accuracy, and precision. Surgical assistance with AR has been shown to improve neuronavigation, enhance surgical planning, and reduce procedure duration in both spinal and cranial neurosurgery. Accordingly, the use of AR in neurosurgery has significantly increased over the last decade [1]. Emerging pre-clinical studies regarding the use of AR in neurosurgery aim to improve neuronavigation, reduce operation times, and improve patient outcomes.

2. Overview of AR

AR is a computer-based technology allowing users to overlay visual information in their field of view by combining real and virtual objects in the real environment, running in real-time, and connecting real and virtual objects [2]. All three conditions are satisfied to create an effective AR system. Unlike Virtual Reality (VR) systems, AR platforms are not fully immersive, but rather a mixture of real and virtual environments. Hardware with the capability to run Augmented Reality Applications (ARAs) include but are not limited to headsets, smart glasses, and mobile devices [3]. These devices use sensors, including cameras, accelerometers, global positioning systems, and solid-state compasses to capture information about the real-world surrounding environment. The processing components of the hardware then transform this information and overlay the appropriate output information to create an augmented view of the user’s visual field [2]. Figure 1 presents a brief overview of how AR has been integrated into the field of neurosurgery.
Figure 1. Brief visual overview of how AR has been integrated into the field of neurosurgery.
Up to this point, AR platforms have been used in a variety of different industries, including the entertainment [4], automotive [5], and healthcare industries. Within healthcare, specialties that require technical skills and dexterity have been more likely to adopt AR systems. Most notably, ARAs have been implemented in dentistry [6], oral and maxillofacial surgery [7], ophthalmology [8], orthopedics, and plastic surgery [9]. In addition, AR platforms have shown to be effective in the treatment of psychological disorders such as phobias and anxiety [2]. In medical education and training, ARAs have been incorporated into teaching anatomy, classroom study aids, and clinical training simulators [8]. For example, due to the high cost and limited availability of cadavers, some medical schools are turning to ARAs to provide foundational anatomic education to medical students. Within the realm of surgery, AR allows students to practice surgical techniques in a highly realistic environment without the need for a physical patient, reducing potential risks [3]. With this reduced risk, students are able to step outside of their comfort zone and attempt more complex surgeries without compromising patient safety. Students can practice using the simulation until they feel more capable in their abilities, and once their skills have progressed, they can begin operating on physical patients.
While AR systems generate a virtual landscape, they are able to provide haptic feedback to trainees, allowing them to develop their skills [9]. AR additionally allows for remote learning and teaching. This process involves an international surgeon using an augmented reality proctoring system where a remote surgeon will assist in the marking of various anatomical landmarks to reduce the risk of complications. One study found that this platform was a safe, reliable, and effective way to remotely conduct cleft deformity repair surgery [10].
Given the increasing prevalence of AR use within various surgical subspecialties, there exists a need to highlight recent applications of AR for intraoperative assistance during cranial and spinal neurosurgery. Furthermore, the potential for AR to be used as a teaching tool demonstrates an additional need to summarize recent advances in AR systems specifically for neurosurgery training programs.

3. Emerging Studies Investigating the Role of AR in Neurosurgery

As demonstrated in the recent literature, AR is rapidly evolving for use in neurosurgical procedures and neurosurgical training. However, despite technological advances in AR software and supporting hardware modalities, few randomized control trials have been conducted to evaluate patient outcomes in large cohorts, including outcomes related to postoperative complications and cost-effectiveness [11]. A large number of studies included in this review tested the use of AR in relatively small sample sizes, indicating that the current body of research regarding the applications of AR in neurosurgery is largely focused on improving and refining AR software and hardware before clinical use in a larger group of patients.
There currently exist two available registered clinical trials investigating the use of AR in neurosurgery (Table 1). One trial at the University of Pennsylvania (NCT03921385) was designed to investigate the safety and efficacy of AR assistance with holographic images projected onto the surgical field for neuronavigation in 32 patients receiving cranial or spinal surgery. This clinical trial was completed on 23 February 2023, though no results have been published in the literature at this time. Another study at Balgrist University Hospital (NCT04610411) is currently recruiting participants for a study aiming to investigate the accuracy of implant navigation using AR glasses to improve pedicle screw and rod implantation (Table 2).
Table 1. Overview pathology, surgical procedure, anatomic location, AR system, outcome, and application of included studies.
Table 2. Emerging AR Studies in Spinal and Cranial Neurosurgery.
Similarly, many reports included in this research detailing the applications of AR for neurosurgical training remain largely in the early stages of development. More specifically, many forms of AR designed to assist with neurosurgical instruction and training currently reported in the literature focus on teaching scenarios in small sample sizes. AR systems designed to aid in surgical instruction in a highly realistic manner and with the ability to account for individual patient differences take a significant amount of time to develop and refine. The studies detailed in this systematic review ultimately suggest that the use of AR in neurosurgical training remains largely in the developmental stages. Once its use been refined, it would be valuable to conduct studies that evaluate the feasibility and efficacy of AR for assistance with neurosurgical training in a large group of neurosurgeons, each with varying skillsets and years of experience. In this way, it would be possible to better understand how AR can be utilized to train a diverse group of trainees to further refine AR-based training modalities.

This entry is adapted from the peer-reviewed paper 10.3390/medicina59101721

References

  1. Cannizzaro, D.; Zaed, I.; Safa, A.; Jelmoni, A.J.; Composto, A.; Bisoglio, A.; Schmeizer, K.; Becker, A.C.; Pizzi, A.; Cardia, A.; et al. Augmented Reality in Neurosurgery, State of Art and Future Projections. A Systematic Review. Front. Surg. 2022, 9, 864792.
  2. Chicchi Giglioli, I.A.; Pallavicini, F.; Pedroli, E.; Serino, S.; Riva, G. Augmented Reality: A Brand New Challenge for the Assessment and Treatment of Psychological Disorders. Comput. Math. Methods Med. 2015, 2015, 862942.
  3. Parsons, D.; MacCallum, K. Current Perspectives on Augmented Reality in Medical Education: Applications, Affordances and Limitations. Adv. Med. Educ. Pract. 2021, 12, 77–91.
  4. Parekh, P.; Patel, S.; Patel, N.; Shah, M. Systematic review and meta-analysis of augmented reality in medicine, retail, and games. Vis. Comput. Ind. Biomed. Art 2020, 3, 21.
  5. Regenbrecht, H.; Baratoff, G.; Wilke, W. Augmented reality projects in the automotive and aerospace industries. IEEE Comput Graph. Appl. 2005, 25, 48–56.
  6. Huang, T.K.; Yang, C.H.; Hsieh, Y.H.; Wang, J.C.; Hung, C.C. Augmented reality (AR) and virtual reality (VR) applied in dentistry. Kaohsiung J. Med. Sci. 2018, 34, 243–248.
  7. Ayoub, A.; Pulijala, Y. The application of virtual reality and augmented reality in Oral & Maxillofacial Surgery. BMC Oral. Health 2019, 19, 238.
  8. Tang, K.S.; Cheng, D.L.; Mi, E.; Greenberg, P.B. Augmented reality in medical education: A systematic review. Can. Med. Educ. J. 2020, 11, e81–e96.
  9. Shafarenko, M.S.; Catapano, J.; Hofer, S.O.P.; Murphy, B.D. The Role of Augmented Reality in the Next Phase of Surgical Education. Plast. Reconstr. Surg. Glob. Open 2022, 10, e4656.
  10. Vyas, R.M.; Sayadi, L.R.; Bendit, D.; Hamdan, U.S. Using Virtual Augmented Reality to Remotely Proctor Overseas Surgical Outreach: Building Long-Term International Capacity and Sustainability. Plast. Reconstr. Surg. 2020, 146, 622e–629e.
  11. Yuk, F.J.; Maragkos, G.A.; Sato, K.; Steinberger, J. Current innovation in virtual and augmented reality in spine surgery. Ann. Transl. Med. 2021, 9, 94.
  12. de Almeida, A.G.C.; Fernandes de Oliveira Santos, B.; Oliveira, J.L.M. A Neuronavigation System Using a Mobile Augmented Reality Solution. World Neurosurg. 2022, 167, e1261–e1267.
  13. Gurses, M.E.; Gungor, A.; Gokalp, E.; Hanalioglu, S.; Karatas Okumus, S.Y.; Tatar, I.; Berker, M.; Cohen-Gadol, A.A.; Ture, U. Three-Dimensional Modeling and Augmented and Virtual Reality Simulations of the White Matter Anatomy of the Cerebrum. Oper. Neurosurg. 2022, 23, 355–366.
  14. Finger, T.; Schaumann, A.; Schulz, M.; Thomale, U.W. Augmented reality in intraventricular neuroendoscopy. Acta Neurochir. 2017, 159, 1033–1041.
  15. Li, C.R.; Shen, C.C.; Yang, M.Y.; Tsuei, Y.S.; Lee, C.H. Intraoperative Augmented Reality in Microsurgery for Intracranial Arteriovenous Malformation: A Case Report and Literature Review. Brain Sci. 2023, 13, 653.
  16. Cabrilo, I.; Bijlenga, P.; Schaller, K. Augmented reality in the surgery of cerebral aneurysms: A technical report. Neurosurgery 2014, 10 (Suppl. S2), 252–260; discussion 260–251.
  17. Rychen, J.; Goldberg, J.; Raabe, A.; Bervini, D. Augmented Reality in Superficial Temporal Artery to Middle Cerebral Artery Bypass Surgery: Technical Note. Oper. Neurosurg. 2020, 18, 444–450.
  18. Sun, G.C.; Wang, F.; Chen, X.L.; Yu, X.G.; Ma, X.D.; Zhou, D.B.; Zhu, R.Y.; Xu, B.N. Impact of Virtual and Augmented Reality Based on Intraoperative Magnetic Resonance Imaging and Functional Neuronavigation in Glioma Surgery Involving Eloquent Areas. World Neurosurg. 2016, 96, 375–382.
  19. Ivan, M.E.; Eichberg, D.G.; Di, L.; Shah, A.H.; Luther, E.M.; Lu, V.M.; Komotar, R.J.; Urakov, T.M. Augmented reality head-mounted display-based incision planning in cranial neurosurgery: A prospective pilot study. Neurosurg. Focus 2021, 51, E3.
  20. Bopp, M.H.A.; Sass, B.; Pojskic, M.; Corr, F.; Grimm, D.; Kemmling, A.; Nimsky, C. Use of Neuronavigation and Augmented Reality in Transsphenoidal Pituitary Adenoma Surgery. J. Clin. Med. 2022, 11, 5590.
  21. Goto, Y.; Kawaguchi, A.; Inoue, Y.; Nakamura, Y.; Oyama, Y.; Tomioka, A.; Higuchi, F.; Uno, T.; Shojima, M.; Kin, T.; et al. Efficacy of a Novel Augmented Reality Navigation System Using 3D Computer Graphic Modeling in Endoscopic Transsphenoidal Surgery for Sellar and Parasellar Tumors. Cancers 2023, 15, 2148.
  22. Dominguez-Velasco, C.F.; Tello-Mata, I.E.; Guinto-Nishimura, G.; Martinez-Hernandez, A.; Alcocer-Barradas, V.; Perez-Lomeli, J.S.; Padilla-Castaneda, M.A. Augmented reality simulation as training model of ventricular puncture: Evidence in the improvement of the quality of punctures. Int. J. Med. Robot. 2023, 19, e2529.
  23. Skyrman, S.; Lai, M.; Edstrom, E.; Burstrom, G.; Forander, P.; Homan, R.; Kor, F.; Holthuizen, R.; Hendriks, B.H.W.; Persson, O.; et al. Augmented reality navigation for cranial biopsy and external ventricular drain insertion. Neurosurg. Focus 2021, 51, E7.
  24. Chiou, S.Y.; Zhang, Z.Y.; Liu, H.L.; Yan, J.L.; Wei, K.C.; Chen, P.Y. Augmented Reality Surgical Navigation System for External Ventricular Drain. Healthcare 2022, 10, 1815.
  25. Montemurro, N.; Condino, S.; Cattari, N.; D’Amato, R.; Ferrari, V.; Cutolo, F. Augmented Reality-Assisted Craniotomy for Parasagittal and Convexity En Plaque Meningiomas and Custom-Made Cranio-Plasty: A Preliminary Laboratory Report. Int. J. Environ. Res. Public Health 2021, 18, 9955.
  26. Van Gestel, F.; Frantz, T.; Buyck, F.; Geens, W.; Neuville, Q.; Bruneau, M.; Jansen, B.; Scheerlinck, T.; Vandemeulebroucke, J.; Duerinck, J. Neuro-oncological augmented reality planning for intracranial tumor resection. Front. Neurol. 2023, 14, 1104571.
  27. Ruggiero, F.; Cercenelli, L.; Emiliani, N.; Badiali, G.; Bevini, M.; Zucchelli, M.; Marcelli, E.; Tarsitano, A. Preclinical Application of Augmented Reality in Pediatric Craniofacial Surgery: An Accuracy Study. J. Clin. Med. 2023, 12, 2693.
  28. Carl, B.; Bopp, M.; Saß, B.; Pojskic, M.; Voellger, B.; Nimsky, C. Spine surgery supported by augmented reality. Glob. Spine J. 2020, 10, 41S–55S.
  29. Liu, A.; Jin, Y.; Cottrill, E.; Khan, M.; Westbroek, E.; Ehresman, J.; Pennington, Z.; Lo, S.L.; Sciubba, D.M.; Molina, C.A.; et al. Clinical accuracy and initial experience with augmented reality-assisted pedicle screw placement: The first 205 screws. J. Neurosurg. Spine 2022, 36, 351–357.
  30. Elmi-Terander, A.; Burström, G.; Nachabe, R.; Skulason, H.; Pedersen, K.; Fagerlund, M.; Ståhl, F.; Charalampidis, A.; Söderman, M.; Holmin, S.; et al. Pedicle Screw Placement Using Augmented Reality Surgical Navigation With Intraoperative 3D Imaging: A First In-Human Prospective Cohort Study. Spine 2019, 44, 517–525.
  31. Butler, A.J.; Colman, M.W.; Lynch, J.; Phillips, F.M. Augmented reality in minimally invasive spine surgery: Early efficiency and complications of percutaneous pedicle screw instrumentation. Spine J. 2023, 23, 27–33.
  32. Kosterhon, M.; Gutenberg, A.; Kantelhardt, S.R.; Archavlis, E.; Giese, A. Navigation and Image Injection for Control of Bone Removal and Osteotomy Planes in Spine Surgery. Oper. Neurosurg. 2017, 13, 297–304.
  33. Hu, M.H.; Chiang, C.C.; Wang, M.L.; Wu, N.Y.; Lee, P.Y. Clinical feasibility of the augmented reality computer-assisted spine surgery system for percutaneous vertebroplasty. Eur. Spine J. 2020, 29, 1590–1596.
  34. Abe, Y.; Sato, S.; Kato, K.; Hyakumachi, T.; Yanagibashi, Y.; Ito, M.; Abumi, K. A novel 3D guidance system using augmented reality for percutaneous vertebroplasty: Technical note. J. Neurosurg. Spine 2013, 19, 492–501.
  35. Carl, B.; Bopp, M.; Sass, B.; Pojskic, M.; Nimsky, C. Augmented reality in intradural spinal tumor surgery. Acta Neurochir. 2019, 161, 2181–2193.
  36. Tigchelaar, S.S.; Medress, Z.A.; Quon, J.; Dang, P.; Barbery, D.; Bobrow, A.; Kin, C.; Louis, R.; Desai, A. Augmented Reality Neuronavigation for En Bloc Resection of Spinal Column Lesions. World Neurosurg. 2022, 167, 102–110.
  37. Molina, C.A.; Dibble, C.F.; Lo, S.L.; Witham, T.; Sciubba, D.M. Augmented reality-mediated stereotactic navigation for execution of en bloc lumbar spondylectomy osteotomies. J. Neurosurg. Spine 2021, 34, 700–705.
  38. Sommer, F.; Hussain, I.; Kirnaz, S.; Goldberg, J.; McGrath, L.; Navarro-Ramirez, R.; Waterkeyn, F.; Schmidt, F.; Gadjradj, P.S.; Hartl, R. Safety and Feasibility of Augmented Reality Assistance in Minimally Invasive and Open Resection of Benign Intradural Extramedullary Tumors. Neurospine 2022, 19, 501–512.
  39. Wanivenhaus, F.; Neuhaus, C.; Liebmann, F.; Roner, S.; Spirig, J.M.; Farshad, M. Augmented reality-assisted rod bending in spinal surgery. Spine J. 2019, 19, 1687–1689.
  40. von Atzigen, M.; Liebmann, F.; Hoch, A.; Miguel Spirig, J.; Farshad, M.; Snedeker, J.; Fürnstahl, P. Marker-free surgical navigation of rod bending using a stereo neural network and augmented reality in spinal fusion. Med. Image Anal. 2022, 77, 102365.
  41. Deib, G.; Johnson, A.; Unberath, M.; Yu, K.; Andress, S.; Qian, L.; Osgood, G.; Navab, N.; Hui, F.; Gailloud, P. Image guided percutaneous spine procedures using an optical see-through head mounted display: Proof of concept and rationale. J. Neurointerventional Surg. 2018, 10, 1187–1191.
  42. Agten, C.A.; Dennler, C.; Rosskopf, A.B.; Jaberg, L.; Pfirrmann, C.W.A.; Farshad, M. Augmented Reality–Guided Lumbar Facet Joint Injections. Investig. Radiol. 2018, 53, 495–498.
More
This entry is offline, you can click here to edit this entry!
Video Production Service