Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1
Mark Southern
 -- 1669 2023-01-30 10:09:04 |
2 format correct
Jessie Wu
 Meta information modification 1669 2023-01-31 02:14:13 | |
3 format correct
Jessie Wu
 -1 word(s) 1668 2023-01-31 02:16:13 | |
4 format correct
Jessie Wu
 + 2 word(s) 1670 2023-01-31 02:17:50 |

Video Upload Options

Do you have a full video?
Send video materials Upload full video

Confirm

Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Bhattacharya, M.;  Penica, M.;  O’connell, E.;  Southern, M.;  Hayes, M. Digital Twin Technology in a Human-in-Loop Setting. Encyclopedia. Available online: https://encyclopedia.pub/entry/40569 (accessed on 07 September 2024).
Bhattacharya M,  Penica M,  O’connell E,  Southern M,  Hayes M. Digital Twin Technology in a Human-in-Loop Setting. Encyclopedia. Available at: https://encyclopedia.pub/entry/40569. Accessed September 07, 2024.
Bhattacharya, Mangolika, Mihai Penica, Eoin O’connell, Mark Southern, Martin Hayes. "Digital Twin Technology in a Human-in-Loop Setting" Encyclopedia, https://encyclopedia.pub/entry/40569 (accessed September 07, 2024).
Bhattacharya, M.,  Penica, M.,  O’connell, E.,  Southern, M., & Hayes, M. (2023, January 30). Digital Twin Technology in a Human-in-Loop Setting. In Encyclopedia. https://encyclopedia.pub/entry/40569
Bhattacharya, Mangolika, et al. "Digital Twin Technology in a Human-in-Loop Setting." Encyclopedia. Web. 30 January, 2023.
Digital Twin Technology in a Human-in-Loop Setting
Edit
This entry is adapted from the peer-reviewed paper 10.3390/systems11010035

The increase in computational capability has led to an unprecedented increase in the range of new applications where machine learning can be used in real time. Notwithstanding the range of use cases where automation is now feasible, humans are likely to retain a critical role in the operation and certification of manufacturing systems for the foreseeable future. Digital twin technology is a virtual representation of a system, updated by real-time data, that uses simulation, machine learning (ML), and reasoning to enhance decision-making. This idea promises to give production systems real-time control.

human-in-loop digital twin data

1. Introduction

Dynamic visualisation in multi-mode is the key requirement of digital twin technology, which offers the user a close approximation of the application domain in real life, and is used for training and problem solving, as given in Figure 1.
Systems 11 00035 g007 550
Figure 1. Three-dimensional visualisation for operator assistance.
Capturing any differences between a simulation and a digital twin (DT) is critical to the success of any use case. The simulation is an offline, conditional experimentation [1], while the latter is a real-time event and the quality of the model determines how accurate any simulation will be. The DT technology explores how the user interaction is captured by the CPS sensors and actuators, and the loss of information between the real and simulated events is kept vanishingly small. With the aid of augmented reality, discrete events can be overlaid with simulation model layouts in real-time over live manufacturing line scenes via headsets and hand devices. Closed loop decision-making that is facilitated using mixed or augmented reality environments is another way in which this information loss can be minimised [2]. Case studies involving virtual reality (VR) presentations of manufacturing settings that are boosted using motion and depth sensors such as Kinect have yielded promising results [3]. Models are built using industrial floor layouts for operator control of discrete event simulation capturing real-time movements and voice commands [4]. Radio frequency identification (RFID) labels are used to monitor and manage logistics on the industry floor, focusing on the visualisation of logistics trajectories [4]. There is a need for research on DT-based manufacturing system feedback control loops [5][6]. DT has also been investigated as a method to control and visualise information flow for more comprehensive product development, using the finished product’s performance as a feedback loop in designing new products [7]. Both larger and small- and medium-sized enterprises (SMEs) can benefit from digital twin technology, as it provides unified data acquisition. Lower price point solutions are emerging in the literature [1][7][8][9]. The need for additional research in DT and visualisation is highlighted to address questions such as:
  • what is the amount of autonomous operation and feedback from the industry floor that will be facilitated through the DT?
  • what modifications may be made to better integrate HIL with DT technologies??
To improve urban planning, building, and service, Dassault created a “Digital Twin Singapore” for civil engineering using its 3D Experience Platform [10].

2. Tools to Model and Manage Digital Twin Applications

The enterprise tools that are now appearing for DT applications can be broadly divided into the platform, simulation, optimisation, diagnostic, and prognosis categories. Across these categories, data quality is the primary driver behind DT fidelity. As displayed in Figure 2, the various categories, namely collection, transmission, storage, processing, fusion, and visualisation, are the tools for managing data in a DT.
Systems 11 00035 g008 550
Figure 2. Tools for data management in a digital twin.
The Thingworx platform [11] has been used to link the DT model to operational products, gather, and display sensor data, and examine results via web applications. Data collection, device administration, big data analysis, industrial protocol translation, and many other services are offered by Thingworx. HIROTEC [12] has also provided case studies in the production of body-in-white closures, exhaust systems, and enclosure manufacturing. Connections between the data from CNC machine operations in real-time and enterprise resource planning system data have been shown to reduce equipment downtime successfully. Siemens has launched the MindSphere platform [13], which provides real-time, secure transmission of data collected from sensors, controllers, and various information systems to the cloud enabling big data analysis and other services. Other data collection tools are also shown in Figure 2. In DT, data transmission needs to be continuous and in real time. Simultaneously, it is also needed to confirm that the data are not missing, the accuracy of the data is maintained to the maximum degree, and that the distance between the real system and DT models remains acceptably small. The container DT model presented by Jedermann et al. [14] provides a state space-based approach to the measurement of this distance. IBM’s Aspera [15] is known for its capacity for fast data transmission of large files over long distances under substandard network speeds. It transmits data using the current WAN infrastructure more quickly than the file transfer protocol (FTP) and hypertext transfer protocol (HTTP). Other tools commonly used for data transmission are shown in Figure 2.
Data storage warrants the safeguarding of data and answers to data access in real time using the read–write mechanism. For instance, Apache HBase is a high-performance, column-oriented, real-time scalable read–write distributed database that is built on the Hadoop software platform. Other data storing systems are shown in Figure 2.
Data processing is the manipulation of data to extract useful information. Apache Spark is an open-source unified analytics engine for real-time data processing and data analytics. Java, Python, and other programming languages are supported by Spark. Data query is also supported using SQL and HiveSQL.
Data fusion combines, correlates, and amalgamates several data sources to provide information that is more reliable, accurate, and practical than information obtained from any one data source alone. In Python, Spyder and Pycharm are used as data fusion software to develop, debug, and as project management, smart prompting, auto-completion, and version control. Other data fusion tools are shown in Figure 2.
The open-source software Echarts provides customised data visualisation for large and dynamic data. Similar tools are shown in Figure 2.

3. Service Applications That Support Human-in-Loop for Digital Twin

Service platform tools fuse emerging technologies such as the Internet of things (IoT), big data, and AI. The application tools include monitoring tools, optimisation tools, diagnostic tools, etc. The diagnosis tools provide intelligent predictive maintenance strategy for equipment and reduce downtime, etc., by analysing and processing the twin data. For example, the ANSYS simulation tool helps to design IIoT-connected devices and data analysis of the connected devices, along with design data for troubleshooting and predictive maintenance purposes [16]. Additionally, data-driven methods can be integrated to determine the remaining tool life to inform the human operator when the replacement of machine parts is needed. Such an example can be found in Baker Hughes, which is an oil industry company. They provide products and services to the oil industries and have developed a predictive alarm system on MATLAB [17].
In an HIL setting, the quality of sensor data, energy costs, and required performance factors are all factors that affect the quality of simulations. Control system operation will only be successful through real-time comparison of simulation and real-world states, measurements, inputs, and outputs. The mitigation of risk factors, reducing energy consumption, and increasing system efficiencies have all been considered in the literature. For example, the Plant Simulation software developed by Siemens is able to optimise the scheduling of the manufacturing line and the layout of the factory [18]. Within the digital twin electric grid, Simulink runs several simulated scenarios and analyses the measured grid data to assess whether the energy reserve is adequate and if the grid controllers need to be altered. Comparable tools are shown in Figure 2. Modern simulation technologies carry out diagnostics, decide the optimal maintenance strategies, and gather data to improve the next-generation design. For example, a lack of appropriate FEM simulation analysis during the design of a CNC machine tool may lead to the failure of the machine under vibration. Alternatively, adding more material to boost strength and lessen vibration raises the price. However, simulating the structure analysis on FEA/FEM in ANSYS and taking into account the performance and strength will meet the design requirements of the CNC machine tools [19].

4. Human-in-Loop as a Human–Machine Interface

In addition to performing autonomous tasks, CPS also provides HIL with physical and cognitive support. This requires a clear bidirectional information flow in form of interactive human–machine interfaces (HMI). Automatic speech recognition, gesture recognition, and augmented reality are the three key elements of HMI in I4.0 [20]. The latter, commonly referred to as virtual reality, is a vital component of the forthcoming I5.0 and digital twin technology. In the augmented age of I5.0, tools such as VR headsets will be used to augment skills and to make processes simpler and more repeatable. A GUI made up of an enhanced reality visualisation environment, a setup panel, and a gesture detection system are combined to create an HMI [21]. The operator gestures are decoded and transmitted by the gesture recognition system, by specifying the control unit’s input signals. Two frameworks are used to build the recognition system. The former is an optical tracking sensor based on stereo vision and the latter uses a step counter, step detector, accelerometer, gyroscope, linear acceleration sensor, and rotation vector included in an android smartphone. The enhanced reality environment then offers the images captured by the cameras along with a simulation environment to observe any location at random and map gestures in the workspace. This visual input from the HIL enables more effective control of the robot because self-collisions or singular configurations can be clearly anticipated [21]. In other studies, the usage of an evolutionary multi-objective and interactive scheduling framework that incorporates the human’s context-aware preferences in the optimisation process is used to implement HMI at the decisional level [22]. A Flexible Job Shop NSGA2 scheduling scheme that represents a number of Pareto optimal scheduling alternatives is provided to the HMI, and the human operator then selects the preferred choice. This helps in equipping the CPS with decision-making capabilities [22][23].
Zolotová et al. [24] present a series of case studies highlighting the potential applications of intelligent solutions using HIL and describes the changing roles of operators in production systems.

References

  1. Lu, Y.; Liu, C.; Kevin, I.; Wang, K.; Huang, H.; Xu, X. Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robot. -Comput.-Integr. Manuf. 2020, 61, 101837.
  2. Turner, C.J.; Hutabarat, W.; Oyekan, J.; Tiwari, A. Discrete event simulation and virtual reality use in industry: New opportunities and future trends. IEEE Trans. -Hum.-Mach. Syst. 2016, 46, 882–894.
  3. Hutabarat, W.; Oyekan, J.; Turner, C.; Tiwari, A.; Prajapat, N.; Gan, X.P.; Waller, A. Combining virtual reality enabled simulation with 3D scanning technologies towards smart manufacturing. In Proceedings of the 2016 Winter Simulation Conference (WSC), Washington, DC, USA, 11–14 December 2016; pp. 2774–2785.
  4. Zhong, R.Y.; Lan, S.; Xu, C.; Dai, Q.; Huang, G.Q. Visualization of RFID-enabled shopfloor logistics Big Data in Cloud Manufacturing. Int. J. Adv. Manuf. Technol. 2016, 84, 5–16.
  5. Negri, E.; Fumagalli, L.; Macchi, M. A review of the roles of digital twin in CPS-based production systems. Procedia Manuf. 2017, 11, 939–948.
  6. Cimino, C.; Negri, E.; Fumagalli, L. Review of digital twin applications in manufacturing. Comput. Ind. 2019, 113, 103130.
  7. Haag, S.; Anderl, R. Digital twin–Proof of concept. Manuf. Lett. 2018, 15, 64–66.
  8. Tao, F.; Cheng, J.; Qi, Q.; Zhang, M.; Zhang, H.; Sui, F. Digital twin-driven product design, manufacturing and service with big data. Int. J. Adv. Manuf. Technol. 2018, 94, 3563–3576.
  9. Uhlemann, T.H.J.; Lehmann, C.; Steinhilper, R. The digital twin: Realizing the cyber-physical production system for industry 4.0. Procedia CIRP 2017, 61, 335–340.
  10. Qi, Q.; Tao, F.; Hu, T.; Anwer, N.; Liu, A.; Wei, Y.; Wang, L.; Nee, A. Enabling technologies and tools for digital twin. J. Manuf. Syst. 2021, 58, 3–21.
  11. Fortino, G.; Guerrieri, A.; Pace, P.; Savaglio, C.; Spezzano, G. IoT Platforms and Security: An Analysis of the Leading Industrial/Commercial Solutions. Sensors 2022, 22, 2196.
  12. Karmakar, A.; Dey, N.; Baral, T.; Chowdhury, M.; Rehan, M. Industrial internet of things: A review. In Proceedings of the 2019 International Conference on Opto-Electronics and Applied Optics (Optronix), Kolkata, India, 18–20 March 2019; pp. 1–6.
  13. Petrik, D.; Herzwurm, G. iIoT ecosystem development through boundary resources: A Siemens MindSphere case study. In Proceedings of the 2nd ACM SIGSOFT International Workshop on Software-Intensive Business: Start-Ups, Platforms, and Ecosystems, Tallinn, Estonia, 26 August 2019; pp. 1–6.
  14. Jedermann, R.; Lang, W.; Geyer, M.; Mahajan, P. Digital Twin Features for the Intelligent Container. In Proceedings of the International Conference on Dynamics in Logistics, Bremen, Germany, 23–25 February 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 217–228.
  15. Labani, M.; Beheshti, A.; Lovell, N.H.; Alinejad-Rokny, H.; Afrasiabi, A. KARAJ: An Efficient Adaptive Multi-Processor Tool to Streamline Genomic and Transcriptomic Sequence Data Acquisition. Int. J. Mol. Sci. 2022, 23, 14418.
  16. Leskovskỳ, R.; Kučera, E.; Haffner, O.; Rosinová, D. Proposal of digital twin platform based on 3D rendering and IIoT principles using virtual/augmented reality. In Proceedings of the 2020 Cybernetics & Informatics (K&I), Velke Karlovice, Czech Republic, 29 January–1 February 2020; pp. 1–8.
  17. Singh, G. Baker Hughes Develops Predictive Maintenance Software for Gas and Oil Extraction Equipment Using Data Analytics and Machine Learning. 2019. Available online: https://www.mathworks.com/company/user_stories/baker-hughes-develops-predictive-maintenance-software-for-gas-and-oil-extraction-equipment-using-data-analytics-and-machine-learning.html (accessed on 30 August 2022).
  18. Zhang, Z.; Wang, X.; Wang, X.; Cui, F.; Cheng, H. A simulation-based approach for plant layout design and production planning. J. Ambient. Intell. Humaniz. Comput. 2019, 10, 1217–1230.
  19. Gupta, A.; Kundra, T. A review of designing machine tool for leanness. Sadhana 2012, 37, 241–259.
  20. Gorecky, D.; Schmitt, M.; Loskyll, M.; Zühlke, D. Human-machine-interaction in the industry 4.0 era. In Proceedings of the 2014 12th IEEE International Conference on Industrial Informatics (INDIN), Porto Alegre, Brazil, 27–30 July 2014; pp. 289–294.
  21. Garcia, M.A.R.; Rojas, R.; Gualtieri, L.; Rauch, E.; Matt, D. A human-in-the-loop cyber-physical system for collaborative assembly in smart manufacturing. Procedia CIRP 2019, 81, 600–605.
  22. Gaham, M.; Bouzouia, B.; Achour, N. Human-in-the-loop cyber-physical production systems control (hilcp 2 sc): A multi-objective interactive framework proposal. In Service Orientation in Holonic and Multi-Agent Manufacturing; Springer: Berlin/Heidelberg, Germany, 2015; pp. 315–325.
  23. Hadorn, B.; Courant, M.; Hirsbrunner, B. Towards Human-Centered Cyber-Physical Systems; Départment d’informatique Université de Fribourg: Fribourg, Switzerland, 2016.
  24. Zolotová, I.; Papcun, P.; Kajáti, E.; Miškuf, M.; Mocnej, J. Smart and cognitive solutions for Operator 4.0: Laboratory H-CPPS case studies. Comput. Ind. Eng. 2020, 139, 105471.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Engineering, Manufacturing
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Mangolika Bhattacharya
,
Mihai Penica
,
Eoin O’Connell
,
Mark Southern
,
Martin Hayes
View Times: 712
Revisions: 4 times (View History)
Update Date: 31 Jan 2023
Table of Contents
    1000/1000
    Hot Most Recent
    ScholarVision Creations
    Feedback