1. The Internet of Telesurgery and Surgical Telementoring
Perhaps the most impressive advancement the technological applications were able to provide in the surgical world is the incurrence of the telesurgical procedures. When discussing the prospects of IoT applications within surgical theaters, there are two key concepts in existence to keep in mind: telesurgery, and telementoring. While telesurgery describes the performance of a surgical operation by a remotely located surgeon, utilizing a network of devices with the aim of transferring the surgeon’s haptic commands to a robotic surgery system, telementoring is the performance of live surgery on-site, with the live assistance of a more experienced surgeon, located off-site
[1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]. In order to achieve the latter, a network of connected cameras, microphones, screens and computers is necessary
[17][18][19][20][21][22][23][25,26,27,28,29,30,31]. A fundamental outline of the network and crosstalk required is depicted in
Figure 1. This connection is mostly provided through the Internet nowadays, and with new and emerging technologies, such as the fifth generation of mobile networks, implementation of teleservices is expected to grow. A 2017 systematic review of the literature on telementoring in the operating room revealed that operating times and complication rates were the same when compared to on-site live mentoring of younger surgeons
[24][32]. This showcases that telementoring possibly has the potential to supplement live surgical training and might allow trainees to even surpass their educational goals when mentored remotely.
Figure 1. Connected IoST entities and workflow of an IoST-based Telementoring/Telesurgery System.
In addition, telementoring has the potential of long-distance consultation with senior and specialized surgeons who can provide live assistance in the operation rather than a simple preoperative consultation, or the costly option of transferring difficult to manage cases in specialized units
[7][8][9][10][11][12][15,16,17,18,19,20]. Systems that are developed for surgical telementoring in live surgery can also be expanded with an array of biosensors for physiological parameter monitoring that is transmitted directly to the mentoring surgeon
[19][27], allowing for the complete monitoring of the surgical patient, rather than just the surgical procedure. Studies evaluating the satisfaction outcomes of surgical trainees whenever live proctoring from a distance was used reveal universally high satisfaction rates
[10][11][12][13][14][15][16][17][18,19,20,21,22,23,24,25] that were common to both mentors and mentees. Participants of said studies felt that telementorship sessions with more experienced surgeons would be a useful addition in surgical education curriculums
[25][26][27][28][29][30][31][32][33,34,35,36,37,38,39,40]. Even when utilized for teleconferencing and simple audiovisual transmission of an operation for teaching purposes, participants rated positively to their experience more than 90% of the time
[25][33][34][35][33,41,42,43]. This goes to show that the introduction of the Internet in the everyday surgical practice can have a pragmatic impact in the education of younger surgeons, particularly at low costs and without complex institutional requirements. Telementoring in surgery allows the safe transfer of knowledge and skill aptitude from a more experienced surgeon to a younger mentee.
Studies reporting telementoring techniques within the OR were split broadly in the two categories of the studied teaching method. The simplest form of surgical telementoring includes the live observation of a surgical procedure by a distant senior surgeon who can provide audiovisual guidance through an Internet connection, or a more advanced form of proctoring including the distant manipulation of a camera-bearing surgical robotic arm.
A study by Hinata et al.
[20][28] compared in a systematic manner the perceived differences in live mentoring with telementoring in robotic surgery. Apart from requiring a stable and fast internet connection being the single drawback, all of the postoperative parameters of the surgical patients were the same between the differently mentored groups in almost all the included studies. In another comparative study of internet-based telementoring versus in-person telementoring, not only were the postoperative results the same between the two groups of mentees, but the mentors also showed a significant trend towards telementoring
[19][27], while in some studies, mentees achieved better postoperative results when distance-mentored
[35][36][37][38][39][40][43,44,45,46,47,48]. In a report on telementored trainees that performed bariatric surgery procedures, those that utilized Internet-based mentoring achieved fewer complications, shorter operative hours, as well as shorter hospitalization time
[41][1]. In addition, Altieri et al. studied whether delivering a surgical skills course over the Internet would be less effective than live mentorship
[21][29]. After the course was completed, the trainees were also found not to perform similarly in the post course assessment, but also showed non-inferior skill decline patterns for a few weeks after the course. These results prove that internet-based mentoring in surgery can be similarly effective in the long-term skill-honing process and is not confined in short-term positive outcomes. Additionally, they found that within every study describing an internet-based telementoring system, there were almost no unexpected intra-operative complications
[39][47]. This proves that the use of the Internet as a mentoring tool can be a safe alternative, even when within a high-risk environment such as the operating room (OR).
With the incurrence of high-speed network connections, the handling of larger data transmissions at no expense of the perceived live mentoring was made possible. Utilizing such connections, several research teams were able to integrate AR within surgical telementoring. In these publications, authors describe the use of specialized AR glasses by the mentees
[2][25][26][27][28][35][36][37][38][39][40][42][43][10,33,34,35,36,43,44,45,46,47,48,49,50]. The glasses were equipped with cameras providing live feeds of the surgical field that were transmitted through the Internet to a distant mentor. The mentoring station on the other end includes motion tracking sensors and cameras that capture the mentor’s movements. These movements are superimposed on the mentee’s field of view and on to the surgical field through the specialized glasses in order for the mentee to closely follow along. The trainees assigned to be mentored with this system achieved higher scores on operational evaluation and were prone to less mistakes
[2][28][35][36][37][38][42][43][44][45][46][10,36,43,44,45,46,49,50,51,52,53]. In one clinical study, AR telementoring was compared with simple telementoring through audiovisual transmission and audio guidance
[36][40][44,48]. People mentored with the AR system made less mistakes and were more accurate. However, in several similar studies, authors noted that mentees on such systems required more time to complete the simulated procedure
[24][26][35][36][32,34,43,44].
More advanced telementoring systems include the ability of the remote mentor to assume control of part of the operating system, most commonly the laparoscope-containing arm of the surgical robot
[32][34][37][39][40][40,42,45,47,48]. Advantages of this setup is the ability to provide direct feedback to the mentee and demonstrate the appropriate handling of the instruments, even though the mentor is located remotely.
Telementoring with the use of Internet connections in surgical specialties, however, is not limited to the operating room. Authors have reported utilizing telementorship to organize skill stations, assess the work of mentee surgeons, broadcast live operations for teaching purposes, teach postgraduate courses, conduct virtual grand rounds, etc.
[25][26][27][28][29][30][31][32][33][43][44][45][46][33,34,35,36,37,38,39,40,41,50,51,52,53]. In all of these activities, the mentees rated their experience almost universally positively. A recent endeavor by Greenberg et al.
[31][39] evaluated a novel surgical skill simulation course that incorporated AR. Post-course evaluation revealed not only satisfied students, but increased aptitude as well. Suzuki et al., developed a surgical training system, using software that could provide an accurate virtual surgical case and robotic controllers that could be connected to the mentee’s PC for a training session. In addition, the software, in addition to the patient models, was available as a cloud-based product, meaning that the exact same simulation can be accessed from anywhere in the world.
Indeed, distance mentoring with the use of the Internet also has certain drawbacks. Firstly, as it is often underlined by authors, all the systems described here require a stable, and, in many instances, high-bandwidth Internet connections. This is often an issue with smaller institutions, or institutions at developing countries. In addition, many of the systems included here, require much more than a webcam and a microphone. Complex setups for surgical telementoring often include robotic systems, expensive software, AR glasses, specialized simulation instruments and more. All the above can prove to be a substantial cost that reaches prohibitive status in certain healthcare systems. Trainees that make use of telementoring systems outside the operating room, have also reported that the experience, although positive, could never replace their presence within the OR
[31][39]. In several studies, authors report one or more instance of technical difficulties with the remote mentoring systems, such as audio or video failure, latency times or interruption of connections
[34][42].
Telesurgery is closely related to another relatively recent advancement in the operating room: robotic surgery. In fact, robotic surgery (the next logical step to laparoscopic surgery) was crucial in planting the idea that if a surgeon can command a machine and perform a procedure from a few meters away, why not apply the same principle in larger distances, perhaps even transatlantic ones. In theory, by utilizing a complex internetworked system of cameras, video streaming, feedback data and data processing and transmission devices, the remotely situated surgeon is able to operate the robotic surgery system and provide real-life, quality surgical outcomes. In this scenario, the IoT concept is mainly applied in the collection, transmission and exploration of the collected data, as well as within the bidirectional transmission of signals. The data processed and transmitted include audio, video, and images. In some experimental systems, where remote control of robotic surgery modalities is involved, the data also include feedback related to the positioning of the robotic arms, the surgeon’s hand positioning and movement. The major impending factor in achieving this, however, was none other than the delay, caused by relaying large amounts of data in a wireless manner over large distances. The first recorded instance of a long-distance surgical procedure was an experimental cholecystectomy performed by the team of Marescaux et al. on swine specimens
[32][40]. In their experimental study, all operations were successful, and the measured latency between the command given by the remotely located surgeon, and the observed response was 155 ms, indicating future perspectives where such surgeries might be part of the everyday practice. The first validation that performing surgical procedures from a distance was feasible and safe came in 2002 from the same research team
[39][47] that performed the first ever cholecystectomy on a real patient, utilizing a robot that was controlled overseas.
Since then, the telesurgery concept has come a long way, with one major milestone being added recently, namely, the incorporation of 5th generation mobile networks in surgical practice. Study groups evaluating the feasibility of remote-controlled robotic surgery systems have found that it is a realistic option for performing surgical operations, with no additional risk for the patient and comparable surgical outcomes
[34][35][36][37][38][39][40][42][43][44][45][46][42,43,44,45,46,47,48,49,50,51,52,53]. The feasibility of remote surgery application is such that surgical teams have achieved successful operations with optimal postoperative results from a remote site of more than 3000 km away from the primary site where the operating suite is located. This means that patients are now able to receive specialized surgery without having to relocate to a specialized surgical center. Utilization of high-speed internet connections is usually in the form of fiber optic connections, or more recently 5G mobile networks, and allow minimal latency within surgical practice, meaning increased safety for the patient as the operator is confident of their movements within the surgical field
[31][39]. Within the included studies, all of the authors report no intra-operative complications, in addition to no further time delays of the surgery. These include further limitation of latency between stations, addressing safety concerns regarding cyber security, as well as the newfound liability due to medical damage caused by remotely operated robotic surgery systems. Financial imbalances between populations and countries are perhaps the greatest obstacle to the popularization of long-distance surgery. However, as the years progress, the availability of stable and fast Internet connections is on the rise and the costs of medical infrastructure, while substantial, can be surpassed by the cost-effectiveness of such advances
[40][48].
Authors have also compared the different options of Internet connections that were applied to telesurgical procedures, namely, land cable connection with satellite connection. No differences in operative parameters such as blood loss or total operational time were found, and the participants felt equally confident with both modalities. In one study, the satellite Internet connection produced significantly greater latency times between the operator and the surgical robot; however, they were not sufficient to constitute the operation as unsafe.
2. Image-Guided Surgery in the IoT Era
The basic principle of Image-Guided Surgery (IGS) is constituted by the utilization of a tracking device, alongside pre or even intra-operative imaging, to aid the surgeon in the spatial orientation during a surgical process. Authors of relevant publications have used IoT networks to incorporate patient imaging, as well as preoperative planning, into the operating room.
The network-based sharing of data and the creation of a workflow through sequential data appraisals and data provision towards the next component makes IGS a prime example of an IoT application in surgery. An IGS system built around the IoT approach usually consists of input of preoperative imaging data of the surgical patient. These data are then used to make reconstructed models of the anatomical area of interest. Such models are then transmitted wirelessly to a modality of choice, ranging from augmented reality (AR) glasses to the viewing screen of a surgical robot
[47][48][49][50][51][52][54,55,56,57,58,59]. A simplified schematic illustrating the workflow of these systems can be seen in
Figure 2, encompassing the idea of the IoT concept defined as a network of inter-connected devices that process and exchange data.
Figure 2. Connected IoST entities and workflow of an IoST-based Image-Guided Surgical System.
Current literature in IGS is heavily referred to as advances in the neurosurgical field. In this aspect, Internet-based IGS has a lot to offer by incorporating data from various sources and making them available to the surgeon in real-time. Augmented reality systems make use of the Internet to transmit preoperative renderings and patient imaging after registering these data with on-site patient images onto specialized smart glasses
[47][48][53][54,55,60]. Coupling preoperative patient imaging with the image-guided system usually involves using certain sensors for the tracking of specific markers within the patient
[48][55]. Studies on real-time surgery indicate that IGS has been very helpful in the identification and preservation of vital structures
[51][52][54][58,59,61]. Almost all of the reports included here report that AR imaging systems allowed the preservation of vital structures in real-life patients
[53][54][55][56][57][60,61,62,63,64], significant accuracy when utilized to assist in biopsies or electrode insertion
[47][48][49][50][51][52][54,55,56,57,58,59] and real-time compensation for brain shift during neurological surgery
[47][54], a novelty not achievable otherwise. In a study by Watanabe et al.
[53][60], the intra-operative tablet devices, used to provide live camera views of the surgical field were also tracked in space by an overhead multi-angle camera system and special tracking spheres
[53][54][55][56][57][60,61,62,63,64]. Internet use within the surgical suit enabled authors to utilize pre- and intra- operative patient imaging to create a live overlay of anatomical areas of interest (such as tumors or sensitive nearby structures) by connecting the preoperative image repository with an image processing software that returned the final image data either to a monitor or another specialized apparatus (e.g., smart glasses)
[52][59].
Eftekhar et al.
[56][63] took the AR integration within the surgical suite a step further by introducing a lesion-tracking smartphone app for mobile phones. The software will utilize the smartphone’s camera to register anatomic landmarks of the patent’s head surface that will be used to align the postoperative imaging with the live image.
Related work by Guo et al.
[58][8] and Li et al.
[47][54] describe the integration of surgical robots within IGS systems. The coupling between the intra-operative MRI imaging, the surgical robot, the central processing unit and the end-display of the processed imaging data is achieved by utilizing wired or wireless Internet connections. Despite lacking trials on their performance in real-life surgery in humans, it is safe to say that the Internet in this case, has provided reliable interconnectivity with multiple appliances at once, and has the potential to become the unseen substrate of modern neurosurgical breakthroughs.
Ushimaru et al.
[59][5] utilized RFID tracking tags placed on laparoscopic instruments in order to track usage in general, as well as the activation times of each tool. Data from the tags were transmitted onto RFID readers within the operating room. The RFID readers then transmitted the recorded data to specialized computer software that converted the electrical current readings to “on/off” indications and activation times. This experimental setup was successful in capturing the usage patterns of surgical instruments during cholecystectomy. This short proof of concept study opens the door towards incorporating network-based tracking systems within the everyday surgical practice that will capture surgical instrument use and possibly aid the surgeon in spatial navigation. Possible implications of this could include coupling a similar tracking system with an AI network that can aid in intra-operative, real-time decision making, or an in-hospital data accumulation system that will be able to measure surgical performance. A research group was recently able to combine the tracking of virtual laparoscopic instruments operated by a distant mentoring surgeon with the live feed image of the laparoscope and the real surgical instruments operated by a mentee
[51][58]. Meier-Hein et al. also managed to construct a tracking algorithm for laparoscopic instruments by using sensor data and an Internet-based integration status. Their system was successful in producing an algorithm that could automatically detect and accurately annotate the laparoscopic surgical instruments, as well as their usage status in real-time operations.
3. The Role of the IoT in Telemonitoring the Surgical Patient
Medical telemonitoring usually consists of a specialized “smart” device that captures target parameters and transmits them through a wireless Internet connection, either directly to the referring physician, or to a centralized repository from which they can be accessed (
Figure 3). The role of the Internet here is more straightforward: instead of being the network substrate that interconnects a variety of operating stations, data repositories and data processing modalities, here, it is used as a unidirectional “data highway” that runs towards the physician. In contrast to previous advances, telemonitoring has been widely implemented in some healthcare systems.
Figure 3. Connected IoST entities and workflow of an IoST-based Telemonitoring System.
Within the surgical patient subgroup, there have been a few clinical studies looking into the applicability of telemonitoring, usually in the postoperative period
[60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75][76][3,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80]. Patients enrolled in an at-home monitoring program after chest wall surgery were also monitored effectively by utilizing the Internet to input certain parameters in an online platform
[61][63][65,67] Cardiac surgery patients were also studied in an IoST rehabilitation program that included wearable biomedical and motion tracking sensors
[64][68]. The physician was therefore able to monitor the patients’ activity levels and their performance in rehabilitation exercises. In a 2021 study by Cos et al., patients scheduled to undergo pancreatic surgery were monitored preoperatively by using a wearable smart device that was able to record heart rate, activity status, etc., and through an internet connection, transmit them to a central server. Not only did patients adhere to this novel concept, but the data that were automatically collected were of such quality that the research team developed an accurate predictive model for postoperative outcomes
[60][3]. Biosensor-based systems are able to wirelessly transmit data on physiological parameters of the patients in order to assist with postoperative monitoring. Authors have reported the incorporation of pulse rate, blood pressure and activity tracking sensors as being successful in monitoring the rehabilitation process of surgical patients
[65][66][67][68][69][70][71][72][73][69,70,71,72,73,74,75,76,77]. Reported advantages, include successful vital signs readings, short training period of nurses and patients alike, less unplanned office visits, and predicting unplanned postoperative complications by indirect monitoring of vital signs
[58][8]. Kim et al. successfully developed a Doppler cuff that could be remotely monitored, allowing remote monitoring of the blood flow of skin flaps. This resulted in superior graft survivability rates
[77][6].
Postoperative monitoring of physiological parameters with the use of biosensors can be further expanded to include automated action after the received sensor signal. In an example by Wang et al.
[71][75], patient-controlled analgesia was administered after indications from a biosensor feedback system that measured physiological responses to pain. Postoperative pain and nausea were reduced in the patients treated with this system. An ostomy alert sensor was developed by the team of Rouholiman et al.
[76][80] that was capable of alerting nurses, patients, and physicians alike of the content status of the ostomy.
The vast array of remote monitoring systems described in the literature could very well be applied to a pre- or post-operative surgical patient, even if there are currently no studies for this specific cohort of patients. Systems used to monitor diseases such as heart failure, hypertension, pregnancy-related complications and more can easily be applied to the surgical patient in the future. A recently developed smartphone application could aid in the distant monitoring of COPD patients and could be useful in the detection of acute exacerbations and advise timely hospitalization
[70][74]. All the above-mentioned modalities for long distance patient monitoring rely on the Internet for data transmission and could very well see their way in surgical patient monitoring or consultation in the not-so-distant future.
There are of course some hurdles still left in the way of universal internet-based patient monitoring. The biggest of which seems to be the reported difficulties elderly patients have with operating such systems
[70][71][72][73][74][74,75,76,77,78]. Technological illiteracy is a persistent issue that seems harder to address than the technicalities of the systems. Patients of older age, of mental burden and patients without a reliable Internet connection that is readily accessible are in danger of being left out of such technological advancements, an observation reported in the majority of clinical studies. What is more, home-based distance monitoring relies entirely upon the adherence of the patient in data recording and the use of the instructed devices.