Most of the studies ^{[1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28]}[58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85] in thereinis review are focused on application of HoloLens 2 in surgical navigation in the operating room and in the emergency department. The use of the AR/MR based system herein in this context provided the user with computer-generated information superimposed to real-world environment and improved accuracy, safety and efficacy of surgical procedures ^[29][86]. The AR-based HoloLens 2 is mainly used as surgical aids aimed at the visualization of medical data, blood vessel search and targeting support for precise positioning of mechanical elements ^[29][86].

In recent years, the Brain-Computer Interface application has been growing rapidly, establishing itself as an emerging technology ^[30][87] tested in several scenarios such as rehabilitation ^[31][88], robotics ^[32][89], precision surgery and speech recognition. However, the usability of many professional brain-sensing equipment remains limited. Indeed, these systems remain expensive, bulky, uncomfortable to wear due to the gel applied to the electrodes and tethered, as well as prone to classification errors. Thus, the modern trend of the scientific community is directed to the use of BCI systems in association with other input modalities such as gaze trackers ^[33][90], or head-mounted display (HMD)s such as Virtual Reality (VR) ^[34][91] and Augmented Reality (AR) headsets ^[35][36][92,93].

Two research contributions ^[37][38][94,95] integrated the BCI and AR HMD systems within the same physical prototype. More specifically, in a first pilot study ^[37][94], it wahe authors proposed a prototype which combines the Microsoft HoloLens 2 with an EEG BCI system based on covert visuospatial attention (CVSA)—a process of focusing attention on different regions of the visual field without overt eye movements. Fourteen participants were enrolled to test the system over the course of two days using a CVSA paradigm. In another study ^[38][95], considering the introduced clip-on solution for the AR-BCI integration, it wahe authors designed a simple 3D game, which changed in real time according to the user’s state of attention measured via EEG and coupled the prototype with a real-time attention classifier. The results of these studies, though promising, needed to be considered preliminary due to the small number of participants (n = 14).

In addition to the described contributions ^[37][38][94,95], the work of Wolf et al. ^[39][96] fits into the human–computer interaction (HCI) field. More specifically, it wahe authors analyzed hand-eye coordination in real-time to predict hand actions during target selection and thus giving the possibility to avoid users’ potential errors before they occur. In a first user study, the authors enrolled 10 participants were enrolled and recorded them playing a memory card game, which involves frequent hand-eye coordination with little task-relevant information. In a second user study, considering a group of 12 participants, the real time effectiveness of the authors’ method to stop participants’ motions in time (i.e., before they reach and start manipulating a target), was evaluated. Despite this contribution’s limitation being represented by the small number of participants, the results demonstrated that the support of the implemented method was effective with a mean accuracy of 85.9%.

Another hot topic in virtual reality research is the use of embodied avatars (i.e., 3D models of human beings controlled by the user), or so-called full-body illusions, a promising tool able to enhance the user’s mental health. To complement theis research, augmented reality is able to incorporate real elements, such as the therapist or the user’s real body, into therapeutic scenarios. Wolf et al. ^[40][97] presented a holographic AR mirror system based on an OST device and markerless body tracking to collect qualitative feedback regarding its user experience. Additionally, authors compared quantitative results in terms of presence, embodiment and body weight perception to similar systems using video see-through (VST), AR and VR. As results, the comparative evaluation between OST AR, VST AR, and VR revealed significant differences in relevant measures (lower feelings of presence and higher body weight of the generic avatar when using the OST AR system).

Augmented reality may be a technology solution for the assessment of gait and functional mobility metrics in clinical settings. Indeed, they provide interactive digital stimuli in the context of ecologically valid daily activities while allowing one to objectively quantify the movements of the user by using the inertial measurement units (IMUs). The project of Koop et al. ^[41][57] aimed to determine the equivalency of kinematic outcomes characterizing lower-extremity function derived from the HoloLens 2 and three-dimensional (3D) motion capture systems (MoCap). Kinematic data of sixty-six healthy adults were collected using the HoloLens 2 and MoCap while they completed two lower-extremity tasks: (1) continuous walking and (2) timed up-and-go (TUG). The authors demonstrated that the TUG metrics, including turn duration and velocity, were statistically equivalent between the two systems.

In the rehabilitation context, the developed technologies such as virtual and augmented reality can also enable gait and balance training outside the clinics. The study of Held et al. ^[42][98] aimed to investigate the manipulation of the gait pattern of persons who have had a stroke based on virtual augmentation during overground walking compared to walking without AR performance feedback. Subsequently, iaut wahors evaluated the usability of the AR feedback prototype in a chronic stroke subject with minor gait and balance impairments. The results provided the first evidence of gait adaptation during overground walking based on real-time feedback through visual and auditory augmentation.

During the COVID-19 pandemic, undergraduate medical training was significantly restricted with the suspension of medical student clerkships onwards. Aiming to continue to deliver training for medical students, augmented reality has started to emerge as a medical education and training tool, allowing new and promising possibilities for visualization and interaction with digital content.

Nine contributions ^{[43][44][45][46][47][48][49][50][51]}[99,100,101,102,103,104,105,106,107] described the use of AR technology and the feasibility of using the HoloLens 2 headset to deliver remote bedside teaching or to enable 3D display for the facilitation of the learning process or in tele-mentoring and tele-consulting contexts.

Wolf et al. ^[43][99], for example, investigated the potential benefits of AR-based and step-by-step contextual instructions for ECMO cannulation training and compare them with the conventional training instructions regularly used at a university hospital. A comparative study between conventional and AR-based instructions for ECMO cannulation training was conducted with 21 medical students. The results demonstrated the high potential of AR instructions to improve ECMO cannulation training outcomes as a result of better information acquisition by participants during task execution.

Several studies ^[49][51][105,107] confirmed that the use of AR technology also enhanced the performance of tele-mentoring and teleconsulting systems in healthcare environments ^[49][105]. Tele-mentoring can be considered as an approach in which a mentor interactively guides a mentee at a different geographic location using a technological communication device.

Bui et al. ^[49][105] demonstrated the usability of AR technology in tele-mentoring clinical healthcare professionals in managing clinical scenarios. In a quasi-experimental study, four experienced health professionals and a minimum of 12 novice health practitioners were recruited for the roles of mentors and mentees, respectively. Each mentee wears the AR headset and performs a maximum of four different clinical scenarios (Acute Coronary Syndrome, Acute Myocardial Infarction, Pneumonia Severe Reaction to Antibiotics, and Hypoglycaemic Emergency) in a simulated learning environment. The role of a mentor, who stays in a separate room, is to use a laptop to provide the mentee remote instruction and guidance following the standard protocols related to each scenario. The mentors and mentees’ perception of the AR’s usability, the mentorship effectiveness, and the mentees’ self-confidence and skill performance were considered as outcome measures.

Bala et al. ^[50][106] presented a proof-of-concept study at a London teaching hospital using mixed reality (MR) technology (HoloLens 2™) to deliver a remote access teaching ward ward-round. ItThe waauthors evaluated the feasibility, acceptability and effectiveness of this technology for educational purposes from the perspectives of students, faculty members and patients.

Four contributions ^{[52][53][54][55]}[108,109,110,111] have been included in the “other applications” subgroup since, due to their characteristics, they cannot be configured as belonging to the subgroups mentioned. More specifically, in the study of Onishi et al. ^[52][108], it wahe authors implemented a prototype system, named Gaze-Breath, in which gaze and breathing are integrated, using an MR headset and a thermal camera, respectively, for hands-free or intuitive inputs to control the cursor three-dimensionally and facilitate switching between pointing and selection. Johnson et al. ^[53][109] developed and preliminarily tested a radiotherapy system for patient posture correction and alignment using a mixed reality visualization. Kurazume et al. ^[54][110] presented a comparative study of two AR training systems for Humanitude dementia care, a multimodal comprehensive care methodology for patients with dementia. HereiIn, it wa this work, authors presented a new prototype called HEARTS 2 consisting of Microsoft HoloLens 2 as well as realistic and animated computer graphics (CG) models of older women. Finally, Matyash et al. ^[55][111] investigated accuracy measurement of HoloLens 2 inertial measurement units (IMUs) in medical environments. Indeed, it wahe authors analyzed the accuracy and repeatability of the HoloLens 2 position finding to provide a quantitative measure of pose repeatability and deviation from a path while in motion.

The number of publications on the Microsoft^® HoloLens 2 application in a medical and healthcare context is shown in Figure 13.

Starting from the year following the release of the second-generation product, the demand for HoloLens 2 has increased exponentially in medical sector until today and the research is expected to expand further in the future. Indeed, in 2020 the number of publications was 3, increasing to 19 in 2021 and to 25 in 2022.

In addition, the use of HoloLens 2 in a medical and healthcare context was analyzed by dividing contributions into the following sub-field applications: surgical navigation, AR-BCI systems integration and human computer interaction, gait analysis and rehabilitation, medical education and training/virtual teaching/tele-mentoring/tele-consulting and other applications. Figure 24 illustrates that surgical navigation represents the most common application (60%, n = 28) of HoloLens 2 and that also in medical training /virtual teaching/tele-mentoring and tele-consulting contexts, the use of this methodology is increasing considerably (19%, n = 9).

Despite the enormous potential of augmented reality in gait analysis and rehabilitation as well as in a brain computer interface, few studies have been published so far (4%, n = 2 and 8%, n = 4).

Concerning the type of publication, most of the reviewed papers were research articles (77%, n = 36), while a smaller percentage (23%, n = 11) was composed of conference proceedings (Figure 35).

Analyzing theour review results in terms of types of visualization technologies (Figure 46), the two types of approaches, AR and MR, were used for applications in a medical and healthcare context. More specifically, AR application was the most common, as evidenced by its use in = 33 research papers (70%), while MR was present in only 14 contributions (30%).

Considering the huge potential application of this technology, also demonstrated in the pandemic context of COVID-19, this systematic literature review aims to prove the feasibility and applicability of HoloLens 2 in a medical and healthcare context as well as to highlight the limitations in the use of this innovative approach and bring focus to emerging research topics, such as telemedicine, remote control and motor rehabilitation.