Components of Common Educational Teleoperation Platform for Robotics: Comparison
Please note this is a comparison between Version 1 by Tero Kaarlela and Version 2 by Jason Zhu.

The erratic modern world introduces challenges to all sectors of societies and potentially introduces additional inequality. One possibility to decrease the educational inequality is to provide remote access to facilities that enable learning and training. A similar approach of remote resource usage can be utilized in resource-poor situations where the required equipment is available at other premises. The concept of Industry 5.0 (i5.0) focuses on a human-centric approach, enabling technologies to concentrate on human–machine interaction and emphasizing the importance of societal values. The novel robotics teleoperation platform supported by the i5.0 reduces inequality and allows usage and learning of robotics remotely independently of time and location. The platform is based on digital twins with bi-directional data transmission between the physical and digital counterparts. 

  • digital twin
  • teleoperation
  • robotics

1. Introduction

Current challenges brought by ongoing conflicts, worldwide threats, and unequal economical conditions [1][2][1,2] are introducing major challenges to all sectors of societies. As a concrete example, pandemic restrictions have set limitations for student group sessions at university laboratories, forcing educational institutes to rethink practical arrangements of laboratory exercises [3][4][5][3,4,5]. Climate change requires an increased appreciation of environmental awareness and preferring green choices [1]. An example of a greener choice is the reduction of traveling to only situations truly necessary and favoring alternative approaches whenever possible. In addition to reduced traveling, changes in consumption habits are required; purchasing of new products should not be preferred, instead leasing or renting of existing equipment should be. Efficient utilization of existing equipment located in remote locations preserves the resources required for manufacturing new units. Equality can be promoted by offering people in developing countries the possibility of training remotely utilizing modern equipment [6][7][6,7]. As Bryndin [8] notes, the aim of Society 5.0 is to create equal possibilities for all individuals and to create an environment for realizing equal possibilities. A possibility to participate from distant locations would provide a solution to overcome pandemic restrictions, enable exercising without traveling, and increase the utilization rate of existing equipment, supporting a human-centric approach of utilizing i5.0 technologies. In this paper, a common teleoperation platform for robotics utilizing digital twins(DT) is presented as a solution for the distant utilization of robotics.
Moniruzzaman et al. [9] defined robotic teleoperation as a robot that is controlled or managed by a human operator over a data connection. Bi-directional communication between the robot and human operator allows the human operator to provide orders and suggestions to the tasks while the robot executes the tasks based on the input from the human operator. In the scope of this proposal, bi-directional communication enables a user to control robots at the university laboratory from any location, therefore saving time and environmental resources consumed on traveling. Teleoperation platform services are also available for students abroad and studying at other universities. There are previous studies on using teleoperations in universities. Marin et al. [10] proposed a telerobotic system, allowing students and scientists to control robots remotely. In addition to this work, the solution presented here provides time resource management to enable the scheduling of teleoperated equipment and Cybersecurity considerations. A trivial calendar application is provided as a front-end for time resource management, allowing students to reserve time resources for a specific robot to exercise robotic skills. In this proposal, the scope of resource management is limited to students reserving time slots to study robotics. The scope can be broadened to include equipment sharing and rental between companies and educational institutes.
Sharing time resources of available equipment requires user authentication and authorization to create a link between time resources, equipment, and user. The platform presented includes methods for user registration, authentication, and authorization. Daily tasks for user registration, strong password creation, and password recovery have been automated to save administrator time.
Social communication is an important feature for students participating in education from distant locations [11][12][11,12]. Communication is needed by students to request help for the usage of the platform and to exchange experiences with fellow students, lecturers, and IT support. On the platform presented, social communication is offered in the form of a live chat, a discussion forum, and an online video meeting. Digital twin (DT) was chosen as a key approach for implementing teleoperation. DT is a digital presentation of a physical world object, serving as a bridge between the physical and cyber world [13]. DTs enable monitoring and operation of physical equipment, production lines, factories, and smart cities independent of location. In this context, DTs enable students to perform practical exercises by teleoperating robots physically located in a university laboratory. The platform includes DTs of mobile, industrial, collaborative, and scara robots for teleoperation. Bi-directional open communication standards are utilized to link digital and physical twins. The digital presentation features a dashboard for controlling and monitoring physical twins. To provide a near real-time view for the teleoperation, a video stream of a physical environment is embedded in the digital presentation.

2. Digital Twin

The concept of Digital Twin (DT) was presented in 2002 by Michael Grieves as a conceptual idea for Product Lifecycle Management [14][17]. Grieves presented virtual and real systems having a link throughout all phases of the product lifecycle from design to disposal. Since then the DT definition has been evolving, and misinterceptions of the “twin” metaphor have existed [15][18]. DIn this paper, definitions presented by Cimino and Kritzinger are considered as guideline definitions for DT. Cimino defines DT as a virtual copy of a physical system able to interact in a bi-directional way [16][19]. According to Kritzinger, bi-directional automated data flow distinguishes DT from the digital model and digital shadow [17][20]. DT has been utilized in the analysis, monitoring, maintenance, engineering, and testing [18][21]. DTs can be used to represent a single system such as a CNC machine, industrial robot, welding equipment, autonomous vehicle, or larger entities such as oil refinery or chemical plant [18][21]. DT enables a user to interact with low-level functions available on the physical part of the twin. A physical twin is an actual system put together to perform a certain function. Digital twin as a digital presentation of physical twin can present physical parameters such as temperatures, linear positions, and vibration frequencies in digital format. DT enables user interaction with the physical twin, providing operator teleoperation capabilities [19][22]. Teleoperation enables operators to control robots over large physical distances [19][20][22,23]. In the context of this paper, DTs enable control of physical robot cells, providing a method for teleoperation. Digital Twin consortium provides glossary [21][24] of DT-related terms. The following terms will be used in our paper later on and are explained here:
  • Digital twin is a virtual presentation of real-world entities and processes, synchronized at specified frequency and fidelity
  • Physical twin is a set of real-world entities and processes that correspond to a digital twin
  • Digital twin platform is a set of integrated services, applications, and other digital twin subsystems that are designed to be used to implement digital twin systems
  • Digital twin system is a system-of-systems that implements digital twin
  • Cyber-Physical system is a system consisting of physical and digital systems integrated via networking.
DT concept is now in its third generation where AI and deep learning algorithms utilize online data [22][23][25,26]. First generation DTs were virtual presentations based on scripting languages and second generation of DTs introduced in 2012 were more simulation-oriented [22][25]. In addition to presenting individual systems, DTs are capable of combining individual DTs to present complete factories, supply chains, airports, and smart cities [18][22][21,25]. International Organization for Standardization (ISO) has defined standards for the DT framework in 2021 [24][27], and standardization is an indication of DT as a mature concept. Deep-learning can enable DTs to self-optimize and thus advance autonomously [22][23][25,26]. A cyber attack against a DT may enable an attacker to gain access to physical twin [25][14]. Therefore, unauthorized modification or destruction of data can lead to unexpected behavior of physical twin, leading to catastrophic damages [25][26][14,28]. According to Holmes et al. [26][28], the amount of CS attacks against DTs are increasing. Integrity and confidentiality of data transfer between physical and digital twins are identified as the main CS challenge for DTs [26][28], as CS attacks disrupting the integrity or confidentiality of data might endanger the safety of the physical twin environment.

Data Flow between Physical and Digital Twin

DT definition requires automated bi-directional data flow between physical and digital parts [17][20]. Synchronized data transfer from individual sensors to digital presentation and from digital presentation to individual actuators is a key enabler of DT concept [16][19]. This enables users to monitor and control physical twin by a user interface of a digital twin. Furthermore, the data flow of individual DTs can be combined to represent complete factories or complete supply chains [22][25]. Connecting individual DTs to present larger entities, a common platform is required. A common platform utilizing open connection standards enables flexible combining of DTs. Currently, OPC Unified Architecture (OPC UA) is a widely adopted open standard cross-platform solution for data exchange [27][29]. OPC UA client-server model has lately been accompanied by publisher–subscriber solutions developed for IoT such as Message Queue Telemetry Transport (MQTT). MQTT, originally developed by Andy Standford-Clark and Arlen Nipper [28][30], is considered one of the most popular messaging protocols for IoT- and IIoT-devices [29][31]. MQTT has been widely adopted in logistics, automotive, manufacturing, and smart home applications. MQTT protocol has three participants: publisher, subscriber, and broker. A publisher is a device publishing certain topics such as temperature and humidity. A subscriber is a device subscribing to certain topics to receive information. Both are connected to a broker to deliver published topics to subscribers [29][31]. MQTT was not originally designed with CS in mind. Current MQTT-versions 3.1.1 and 5 offer authentication as a measure against CS attacks. To encrypt MQTT messaging Transport Layer Security (TLS) is needed and is supported by some MQTT implementations [29][31]. Authentication and encryption provide strong protection against CS attacks. MQTT meets requirements F1, F2, and N3 of digital twins, teleoperation, and CS. OPC Unified Architecture is a cross-platform open standard IEC62541 for data exchange. The standard is actively developed by independent committee OPC Foundation [30][32]. The history of OPC (OLE For Process Control) began when Microsoft introduced the BackOffice suite of server products in 1996 [31][33]. OPC UA standard defined in 2006 is widely recognized in industrial automation and has been chosen as Industry 4.0 reference standard [27][32][29,34]. In 2018, part 14 was added to define publisher/subscriber communication model in addition to client/server model [33][35]. MQTT, AMQP, and UADP messaging protocols are currently supported to offer publisher/subscriber communications to OPC UA. OPC UA standard is secure-by-design, providing confidentiality and integrity by signing and encrypting messages [25][14]. Basic security provided by username and password authentication can be extended with encryption to enable a high level of CS.

3. Teleoperation

Teleoperation of robots has been studied for decades since the beginning of robotic systems in the 1950s [34][36]. MIT started to study teleoperation in the mid-1960s and the mid-1990s when prof. Sheridan reported progress in the field of teleoperation in several countries and application areas including space, undersea, mines, toxic waste cleanup, telediagnosis, and telesurgery [35][36][37,38]. Since then, teleoperation applications have been researched for healthcare [20][37][38][23,39,40], industrial [39][41], education [5][10][5,10], underwater [40][42], nuclear [41][43], and energy applications [42][44]. González et al. [39][41] proposed a teleoperation system to control industrial robots. The system proposed allows the operator to perform finishing, sanding, deburring, and grinding operations, which are hard to do with industrial robots. By using teleoperation it is possible to preserve the physical integrity of human work and to allow people with motor disabilities to perform grinding and sanding processes. A teleoperated robotic system for healthcare performing ultrasound scanning was presented by Duan et al. [37][39]. Tests were conducted and proven that using teleoperated robots to take ultrasound was safe and effective [37][39]. Caiza et al. [42][44] used the lightweight protocol MQTT for the teleoperation of robots. An operator can be in a different location and control the robot movements and actions remotely. In the application proposed by Caiza et al., an operator can control a mobile manipulator to perform inspections on oil and gas equipment [42][44].

Real-Time Video

Low latency of real-time video transmission is critical in teleoperation applications [43][45]. Delay in video transmission has a negative impact on user experience and may cause the user to misguide teleoperated equipment. Web Real-Time communication project (WebRTC) is an open-source project offering near real-time web-based communication [44][46]. Most modern web browsers support WebRTC, and Round Trip Times of 80 to 100 milliseconds have been measured on mobile platforms [45][47]. WebRTC utilizes User Datagram Protocol (UDP) for video streaming. Since UDP does not support congestion control, a custom congestion control algorithm is required to control the video stream.

4. Authentication and Authorization

Methods to authorize platform users and define user permissions are needed. Most modern CMSs’ provide described functionality and also tools for website content management [46][48]. CMS provides schemes and front-end for user registration, strong password creation, password authentication, and password recovery. Currently, three major open-source CMS solutions are WordPress, Joomla, and Drupal [47][49]. A wide variety of third-party plug-ins are available for mentioned three major CMS. Plug-ins can add features such as calendars, clocks, and photo galleries to CMS [46][48]. To allow registered users to log in to all services on the platform with a single username and password, a single sign-on (SSO) method is required. Single sign-on allows registered users to log in once and access all services on the platform. Open Authorization 2.0 (OAuth 2.0) is a token-based open standard for authentication and authorization [48][50]. OAuth 2.0 is a widely utilized standard in Internet communications utilized by Google, Amazon, and Paypal [49][51]. Traditional server–client style authentication is based on sharing credentials between resource owner and client. Sharing credentials with clients can lead to password leaks, data breaches, and unwanted access to protected resources [48][50]. OAuth 2.0 differs from traditional server–client style authentication by passing access tokens instead of credentials from the authentication server to the client. OAuth 2.0 enables users to utilize all services on the platform by creating single credentials on Drupal CMS.
Video Production Service