Augmented Reality in Neurosurgery

Augmented Reality in Neurosurgery: Comparison

Please note this is a comparison between Version 1 by Brandon Lucke-Wold and Version 3 by Catherine Yang.

Augmented reality (AR) involves the overlay of computer-generated images onto the user’s real-world visual field to modify or enhance the user’s visual experience. With respect to neurosurgery, AR integrates preoperative and intraoperative imaging data to create an enriched surgical experience that has been shown to improve surgical planning, refine neuronavigation, and reduce operation time. In addition, AR has the potential to serve as a valuable training tool for neurosurgeons in a way that minimizes patient risk while facilitating comprehensive training opportunities.

augmented reality
machine learning
neurosurgery
cranial neurosurgery

1. Introduction

AR technology involves the superimposition of computer-generated images onto the user’s real-world visual field to modify or enhance the user’s visual experience. In medicine, AR has been utilized in various surgical subspecialties, including ophthalmology, general surgery, orthopedic surgery, urology, and oral and maxillofacial surgery for its ability to integrate computer-generated images with intraoperative visualization of the surgical field [1]. The field of neurosurgery in particular has been at the forefront of image-guided surgery since the early 1990s, as surgeons must demonstrate high intraoperative accuracy and precision to navigate microscopic targets in a way that maximizes patient outcomes while also preserving neurologic functioning. Conventional neuronavigation and neuroimaging technologies involve the use of two-dimensional images to guide surgical planning and neuronavigation. However, integration of preoperative and/or intraoperative magnetic resonance imaging (MRI), computerized tomography (CT), angiography, and tractography images into the real-world surgical environment provides neurosurgeons with an enriched, three-dimensional (3D), semi-immersive experience designed to improve surgical planning, accuracy, and precision. Surgical assistance with AR has been shown to improve neuronavigation, enhance surgical planning, and reduce procedure duration in both spinal and cranial neurosurgery. Accordingly, the use of AR in neurosurgery has significantly increased over the last decade [1]. Emerging pre-clinical studies regarding the use of AR in neurosurgery aim to improve neuronavigation, reduce operation times, and improve patient outcomes.

2. Overview of AR

AR is a computer-based technology allowing users to overlay visual information in their field of view by combining real and virtual objects in the real environment, running in real-time, and connecting real and virtual objects [2]. All three conditions are satisfied to create an effective AR system. Unlike Virtual Reality (VR) systems, AR platforms are not fully immersive, but rather a mixture of real and virtual environments. Hardware with the capability to run Augmented Reality Applications (ARAs) include but are not limited to headsets, smart glasses, and mobile devices [3]. These devices use sensors, including cameras, accelerometers, global positioning systems, and solid-state compasses to capture information about the real-world surrounding environment. The processing components of the hardware then transform this information and overlay the appropriate output information to create an augmented view of the user’s visual field [2]. Figure 1 presents a brief overview of how AR has been integrated into the field of neurosurgery.

Figure 1.

Brief visual overview of how AR has been integrated into the field of neurosurgery.

Up to this point, AR platforms have been used in a variety of different industries, including the entertainment [4], automotive [5], and healthcare industries. Within healthcare, specialties that require technical skills and dexterity have been more likely to adopt AR systems. Most notably, ARAs have been implemented in dentistry [6], oral and maxillofacial surgery [7], ophthalmology [8], orthopedics, and plastic surgery [9]. In addition, AR platforms have shown to be effective in the treatment of psychological disorders such as phobias and anxiety [2]. In medical education and training, ARAs have been incorporated into teaching anatomy, classroom study aids, and clinical training simulators [8]. For example, due to the high cost and limited availability of cadavers, some medical schools are turning to ARAs to provide foundational anatomic education to medical students. Within the realm of surgery, AR allows students to practice surgical techniques in a highly realistic environment without the need for a physical patient, reducing potential risks [3]. With this reduced risk, students are able to step outside of their comfort zone and attempt more complex surgeries without compromising patient safety. Students can practice using the simulation until they feel more capable in their abilities, and once their skills have progressed, they can begin operating on physical patients. While AR systems generate a virtual landscape, they are able to provide haptic feedback to trainees, allowing them to develop their skills [9]. AR additionally allows for remote learning and teaching. This process involves an international surgeon using an augmented reality proctoring system where a remote surgeon will assist in the marking of various anatomical landmarks to reduce the risk of complications. One study found that this platform was a safe, reliable, and effective way to remotely conduct cleft deformity repair surgery [10]. Given the increasing prevalence of AR use within various surgical subspecialties, there exists a need to highlight recent applications of AR for intraoperative assistance during cranial and spinal neurosurgery. Furthermore, the potential for AR to be used as a teaching tool demonstrates an additional need to summarize recent advances in AR systems specifically for neurosurgery training programs.

3. Emerging Studies Investigating the Role of AR in Neurosurgery

As demonstrated in our review of the recent literature, AR is rapidly evolving for use in neurosurgical procedures and neurosurgical training. However, despite technological advances in AR software and supporting hardware modalities, few randomized control trials have been conducted to evaluate patient outcomes in large cohorts, including outcomes related to postoperative complications and cost-effectiveness ^[11][56]. A large number of studies included in this review tested the use of AR in relatively small sample sizes, indicating that the current body of research regarding the applications of AR in neurosurgery is largely focused on improving and refining AR software and hardware before clinical use in a larger group of patients. There currently exist two available registered clinical trials investigating the use of AR in neurosurgery (Table 1). One trial at the University of Pennsylvania (NCT03921385) was designed to investigate the safety and efficacy of AR assistance with holographic images projected onto the surgical field for neuronavigation in 32 patients receiving cranial or spinal surgery. This clinical trial was completed on 23 February 2023, though no results have been published in the literature at this time. Another study at Balgrist University Hospital (NCT04610411) is currently recruiting participants for a study aiming to investigate the accuracy of implant navigation using AR glasses to improve pedicle screw and rod implantation (Table 2).

Table 1.

Overview pathology, surgical procedure, anatomic location, AR system, outcome, and application of included studies.

Source	Pathology or Surgical Procedure	Description of AR System	Outcome

Study	Study Type	Title	Description
Intraoperative Assistance for Cranial Neurosurgery
NCT04610411 Recruiting Zurich, Switzerland	Interventional	Evaluation of the Accuracy of Surgical Navigation in Spine Instrumentation Using Augmented Reality	Commercially available AR glasses are used during pedicle screws and rod implants to investigate the accuracy of implant navigation.
Almeida et al. ^[12][13]	Various cerebral targets	AR created with Swift combined CT images with real-time scans of the skull intraoperatively. AR-identified structures were projected onto a mobile device.	Surgical target recognition with a mean target error of 2.6 ± 1.6 mm.
NCT03921385 Closed Philadelphia, Pennsylvania, USA	Observational	Augmented Reality Enhanced Surgery: Proof-of-concept Study for the Use of Holographic Technology in Cranial and Spinal Surgery	Studying the feasibility of integrating holographic technology as a heads-up display for navigated spinal and cranial neurosurgical procedures.	Gurses et al. ^[13][15]	Various white-matter tracts within the brain	Step-by-step dissection of cadaveric brains was used to created 2D and 3D AR models of white-matter tracts within the brain.	2D and 3D models were successfully obtained. Models could be freely rotated in various planes allowing for complete visualization of white-matter tracts.
Finger et al. ^[14][16]	Right parietal lesion	Scopus NovaPlan navigation software combined with preoperative CT images were injected and registered onto the preoperative microscope to guide surgical trajectory.	Significant reduction in lesion volume of 47%; 1.2 ± 0.4 mm deviation from target.
Li et al. ^[15][18]	Right parietal AVM with evidence of rupture	Microresection of AVM was conducted with an AR holographic projection of angiographic architecture projected onto the cortical surface.	Patient tolerated procedure well and remained stable 6 months after surgery.
Cabrilo et al. ^[16][19]	Unruptured aneurysm	Preoperative 3D MRI, CT, and angiography images were used to create virtual segments of individual patient angiographic architecture. These images were injected into the operative microscope.	Continuous monitoring of surgical accuracy was achieved with optimal clipping and minimized exposition.
Rychen et al. ^[17][20]	Superficial temporal artery to middle cerebral artery extracranial-intracranial (EC-IC) bypass revascularization surgery	Preoperative 3D images of the STA, middle meningeal artery, and primary motor cortex were segmented and injected into the surgical microscope. These images were projected onto the surgical field intraoperatively.	AR was found to improve neuronavigation.
Sun et al. ^[18][21]	Glioma	Preoperative MRI images were obtained and projected onto the surgical microscope intraoperatively.	Tumor resection rate was significantly increased in patients receiving AR assisted surgery (p < 0.01).
Ivan et al. ^[19][22]	Elective craniotomy for various cerebral tumors	Circumferential tumor border tracing and incision planning was conducted with HoloLens AR glasses using preoperative MRI images.	Five procedures were rated as having an excellent correspondence degree, five had an adequate correspondence degree, and one had poor correspondence.
Bopp et al. ^[20][25]	Pituitary lesions requiring transsphenoidal surgery	Preoperative CT images were injected onto the heads-up display intraoperatively.	Use of AR significantly improved surgical accuracy.
Goto et al. ^[21][26]	Sellar and parasellar tumors requiring transsphenoidal surgery	Preoperative radiographic data was superimposed onto the surgical field intraoperatively.	The AR navigation system was found to improve patient outcomes in a majority of patients.
AR as a Training Tool for Cranial Neurosurgery
Dominguez-Valasco et al. ^[22][27]	Ventricular puncture on a skull phantom	AR was designed to project 3D images of human anatomy on a 3D model of the head.	Enhanced accuracy was achieved for 48 neurosurgeons using AR assistance.
Skryman et al. ^[23][28]	Cranial biopsy and EVD insertion on a 3D-printed skull phantom	Intraoperative CT images were combined with integrated video tracking to create an AR system to guide cranial biopsies and EVD insertion.	Mean needle biopsy accuracy was 0.8 mm ± 0.43 mm. Median needle device insertion time was 149 s. Mean EVD insertion accuracy was 2.9 mm ± 0.8 mm. Median insertion time of EVD was 188 s.
Chiou et al. ^[24][29]	EVD insertion	Surgical target position, scalpel entry point, and scalpel direction was displayed on a tablet and superimposed on a phantom patient’s head.	High visual acuity was achieved with AR assistance.
Montemurro et al. ^[25][30]	En plaque cranial vault meningioma located in the left frontotemporal region	Preoperative MRI images were obtained to generate a head-mounted AR system projected onto a patient-specific mannequin before surgical intervention.	Bone flap tracing ang surgical trajectory planning was achieved with an error of less than ±1 mm.
Van Gestel et al. ^[26][31]	Intracranial tumor resection	HoloLens AR was used to determine tumor borders for optimal surgical planning.	registration errors remained below 2.0 mm. tumor delineation was deemed superior in 65% of cases. Surgical planning time was significantly reduced overall.
Ruggiero et al. ^[27][32]	Pediatric craniofacial surgery	HoloLens AR was used to determine tumor borders for optimal surgical planning.	Surgical trajectory was successfully traced with an accuracy level of ±1.5 mm.
Intraoperative Assistance for Spinal Neurosurgery
Carl et al. ^[28][34]	Intradural tumor resection	AR system was designed from preoperative images and injected into the surgical microscope and on a heads-up display.	AR was successfully applied for all cases and visualization of structures was significantly improved with AR.
Liu et al. ^[29][36]	Pedicle screw placement in the thoracic, lumbar, and sacral spine	Three-dimensional images obtained from the O- and C-arm intraoperatively were combined to create a head-mounted AR system.	AR assistance provided an overall surgical accuracy of 98.0%.
Terander et al. ^[30][37]	Pedicle screw placement in the thoracic, lumbar, and sacral spine	Three-dimensional images obtained from 4 cameras in the C-arm intraoperatively were combined to create a head-mounted AR system.	AR assistance provided an overall surgical accuracy of 94.1%.
Butler et al. ^[31][38]	Degenerative spinal pathology, tumor, and spinal deformity	AR system featured a wireless headset with a transparent near-eye display that projected intraoperative 3D images onto the retina of the surgeon to create a “see-through” surgical field.	Pedicle screw placement time was significantly reduced with AR assistance. Screw placement accuracy was significantly improved with AR assistance.
Kosterhon et al. ^[32][39]	Osteotomy in the thoracic and lumbar spine	Three-dimensional virtual resection places were created from preoperative CT images and injected into the surgical microscope.	Patient tolerated the procedure well and experienced no complications.
Hu et al. ^[33][41]	Percutaneous vertebroplasty	Three-dimensional images from preoperative CT scans were superimposed onto the surgical field.	Operative time (p < 0.001) and surgical accuracy (p = 0.028) were significantly improved with AR assistance.
Abe et al. ^[34][42]	Percutaneous kyphoplasty in the thoracic, lumbar, and sacral spine	Three-dimensional images from preoperative CT scans were superimposed onto the surgical field via a head-mounted display.	Improved needle projection planning was seen in forty phantom trials and five patients with no significant complications.
Carl et al. ^[35][44]	Intradural spinal tumors in the cervical, thoracic, lumbar, and sacral spine	AR system was designed from preoperative CT and MRI images and injected into the surgical microscope and on a heads-up display.	Visualization of anatomical structures was significantly improved with AR. Operation time was significantly reduced with AR assistance.
Trigchelaar et al. ^[36][45]	En bloc resection of spinal column lesions	AR system was designed to display augmented images of spinal anatomy within the operative microscope.	Intraoperative use of this AR system resulted in enhanced neuronavigation, smaller surgical incisions, and reductions in bony resection extent with no known patient complications.
Molina et al. ^[37][46]	En bloc lumbar spondylectomy osteotomy of chordoma	Three-dimensional images from preoperative CT scans were superimposed onto the surgical field via a head-mounted display.	Patient tolerated procedure well with no postoperative complications.
Sommer et al. ^[38][48]	Surgical resection of benign intradural extramedullary tumors	Three-dimensional images from preoperative CT and MRI scans were fused and superimposed onto the surgical field.	This AR system was able to identify tumor location and margins. No postoperative complications were observed.
AR as a Training Tool for Spinal Neurosurgery
Wanivenhaus et al. ^[39][50]	Rod manipulation of the cervical, thoracic, lumbar, and sacral spine	HoloLens AR was used to determine spine morphology to guide rod manipulation in a phantom patient.	Rod manipulation was feasible in a phantom patient.
Atzigen et al. ^[40][51]	Rod manipulation of the cervical, thoracic, lumbar, and sacral spine	HoloLens AR was used to determine spine morphology to guide rod manipulation in human cadavers.	Rod manipulation was feasible in a human cadaver and operation time was reduced with AR assistance.
Deib et al. ^[41][54]	Vertebroplasty, kyphoplasty, and percutaneous discectomy	HoloLens AR was used to project spinal anatomy on a phantom patient.	Intraoperative navigation was improved, and operation times were reduced in trainees using HoloLens AR.
Agten et al. ^[42][55]	Lumbar facet joint injections	HoloLens AR was used to project CT images of spinal anatomy on a phantom patient.	HoloLens AR developed from CT images of the spine has an injection accuracy of 97.5% compared to injections guided with CT images alone, which demonstrated an accuracy of 100% in a spine phantom.

Table 2.

Emerging AR Studies in Spinal and Cranial Neurosurgery.

Table generated using information from clinicaltrials.gov.

Similarly, many reports included in this resviearchw detailing the applications of AR for neurosurgical training remain largely in the early stages of development. More specifically, many forms of AR designed to assist with neurosurgical instruction and training currently reported in the literature focus on teaching scenarios in small sample sizes. AR systems designed to aid in surgical instruction in a highly realistic manner and with the ability to account for individual patient differences take a significant amount of time to develop and refine. The studies detailed in this systematic review ultimately suggest that the use of AR in neurosurgical training remains largely in the developmental stages. Once its use been refined, it would be valuable to conduct studies that evaluate the feasibility and efficacy of AR for assistance with neurosurgical training in a large group of neurosurgeons, each with varying skillsets and years of experience. In this way, it would be possible to better understand how AR can be utilized to train a diverse group of trainees to further refine AR-based training modalities.