You're using an outdated browser. Please upgrade to a modern browser for the best experience.
Remote Maintenance Based on Augmented Reality: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Contributor: Dimitris Mourtzis , Vasileios Siatras , John Angelopoulos

Equipment maintenance is one of the key elements consisting of manufacturing systems. Under the framework of Industry 4.0 in an attempt to bridge the physical world with virtual worlds, new technologies have emerged, such as Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR).

  • augmented reality
  • maintenance
  • real-time

1. Introduction

Equipment maintenance is one of the key elements consisting of manufacturing systems. In their research work, Mourtzis et al. in Reference [1] highlight the importance equipment maintenance as part of the production lifecycle, which reaches 60%–70% of the total production cost. Therefore, being able to forecast machine maintenance operations and perform them in a short time period, can lead to successful troubleshooting and simultaneously increase machine tools availability. Additionally, since the replacement of damaged components can be as high as 70% of the total maintenance cost [1], it is one of the top priorities among manufacturing firms to discover alternative policies for cutting maintenance costs as a means of increasing their revenues. As a result, the overall production performance is optimized [2].
Besides the failure forecast, a major issue identified is the completion of accurate and error-free maintenance operations, and, thus, the reassurance of fully functional machines in the earliest time possible. Based on that concept, a considerable amount of research effort has been put on the design and development of real-time maintenance support tools and applications based on mobile devices to prevent unnecessary errors from happening [3][4][5][6]. With the evolution of Information and Communication Technologies (ICT) as well as the massive break out of augmented reality (AR), the research community extended that approach by exploiting the advantages of AR for data projection during the maintenance operations [6][7][8][9][10][11]. Generally, under the framework of Industry 4.0 in an attempt to bridge the physical world with virtual worlds, new technologies have emerged, such as Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) [12]. These digital technologies lie under the umbrella term Extended Reality (XR). The term XR is often used by practitioners and scientists in order to describe the three immersive technologies mentioned above [13].
However, most of the research works available on the web cover basic aspects of Maintenance and Repair Operations (MRO) and provide flexible and accurate troubleshooting [3][4][5][6]. The constant innovation of cutting-edge technologies unveils new opportunities for remote maintenance support. Therefore, there is an apparent need for the development of real-time AR frameworks for the support of MRO.
More specifically, the proposed research deals with the design and development of a novel framework for real-time remote maintenance support based on AR. Inspired by the concepts of Cloud manufacturing and Product Service Systems (PSS), the framework was built in a cloud-enabled environment aimed at bridging the different working groups involved in the production lifecycle. Considering that companies seek increased operational availability, also known as availability (A) of their machines, engineers are constantly developing new maintenance policies that can ensure decreased Mean Time To Repair (MTTR) and increased Actual Machining Time (AMT). The following formula reflects the correlation between the above-mentioned variables and characteristics of actual machines.
R O T B F = M T B F − A M T

where ROTBF is the Remaining Operating Time Between Failures. MTBF is the Mean Time Between Failures. AMT is the Actual Machining Time.

A = M T B F M T T R + M T B F
where A is the machine availability. MTTR is the Mean Time To Repair.
From Equation (2), it is clear that, in order to increase the operational availability of the machine, represented by A in Equation (2), there are two possible alternatives. The first alternative is to increase the numerator, i.e., the MTBF, and the second alternative is to decrease the denominator, i.e., (MTTR+MTBF). However, under the scope of this research work, the aim is to develop a framework that will facilitate the maintenance process of machine tools and, by extension, it will result in the minimization of MTTR. Therefore, if MTTR is decreased, then the denominator of Equation (2) will also decrease and, thereby, the operational availability of the machine tool will eventually increase.
It is stressed out that the above-mentioned indexes are the cornerstones of maintenance contracts offered either as a service by maintenance contractors or OEMs (Original Equipment Manufacturers) as part of their sales program. Although, a maintenance plan may seem attractive due to the low MTTR offered by the maintenance contractor, e.g., 24 h. The Mean Time To Recovery is another index that is highly affecting the quality of the service, as it directly affects Operational Availability of the machine. The Mean Time To Recovery is an aspect of technical maintenance, dependent on a variety of factors, such as part lead times, administrative delays, and technician transportation delays. Aimed at minimizing the MTTR and eliminating the factors mentioned above, this research work is dedicated to the modeling, development, and practical implementation of a real-time, AR-based framework for remote maintenance support.

2. Maintenance as Part of the Modern Manufacturing System

As stated in the previous paragraphs, it is clear that maintenance besides being a crucial part of the production line, as it ensures the continuous and flawless operation of the physical assets, is also a very complex process. Therefore, there is a considerable amount of research work trying to cover all of its aspects. However, maintenance activities can be further categorized regarding the status of the physical assets. Concretely, the most common types of technical maintenance found are: (i) Corrective Maintenance, (ii) Preventive Maintenance, (iii) Risked-based Maintenance, and (iv) Condition-based Maintenance [14].
Throughout recent decades, many types of technical maintenance policies have been developed in an aim to cover the majority of the industrial needs. However, with the big diversity in the general policies that firms adopt, there is no optimal solution regarding the equipment maintenance plan. With that said, the maintenance policy adopted a company, and is highly dependent on the general business model and, by extension, to the general goals to be achieved. As a result, all of the above-mentioned policies are equally important for the research community and the industrial domain as well. Regardless, the scope of this research work is to present the design and development of a framework that supports field technicians mainly in corrective scenarios, where a malfunction has already occurred and, in many cases, standard procedures cannot be applied. Therefore, the research work here could be classified as an enhanced corrective maintenance policy, which is based on the provision of service for on-demand guidance.
The current industrial era is characterized by the immense decentralization of the manufacturing networks. Thus, the actual manufacturing plant may be in a different region than the firm’s headquarters, which makes communication of the two departments more difficult [3][15]. However, the ongoing technological advances in the ICT have enabled the remote monitoring, remote operation, and the remote maintenance of the modern manufacturing systems [16][17]. Modernized maintenance under the term of Smart Maintenance have occurred over the last decade.

3. Augmented Reality Based Remote Maintenance

The concept of Digital Reality has been under the research scope during recent decades. With the advent of the fourth industrial revolution, such cutting-edge digital technologies have met an increasing advance, as a result of the technological advances in the ICT. More specifically, Augmented Reality (AR) is a very popular digital technology, which offers the advantage of enhancing the user’s perception by augmenting their physical environment with computer-generated information. In the field of AR, Azuma is considered a pioneer, with the first survey on AR being published two decades ago [18], concluding that AR back then was a very immature technology. A huge potential for future development was expected. In a more recent publication [19], the challenges that must be overcome by researchers in order to make AR a mainstream form of media are discussed. Among others, the most important challenge the research and development community have to overcome, is the creation of highly intuitive AR applications and tools.
In the available literature, there can be found many publications covering the AR-based aspect of maintenance. The majority of the works is focused on the remote assistance of technical maintenance operations through the various AR-based frameworks. A typical application of remote maintenance support is presented in Reference [20], where the technicians are capable of performing maintenance operations in robotic arm manipulators with the use of AR-projected instructions. Another unique AR framework for the support of MRO is presented in Reference [21]. In Reference [22], the authors cover another aspect of AR-based remote maintenance under the framework of Product Service Systems (PSS). Therefore, the AR tools developed can facilitate manufacturers to add value to their line of products by transitioning to the PSS philosophy.
In the research work presented in Reference [4], the authors have carried out research on the recent trends around the Smart Maintenance paradigm by investigating recent publications as well as by extracting useful empirical knowledge. In Reference [5], a collaborative remote maintenance framework has been suggested. Although the research work is inspiring and of good quality, the framework implementation is based on technologies and equipment that currently can be considered as obsolete, as the technician has limited mobility since a PC is required by their side so as to visualize the AR instructions. Moreover, the instructions are based mostly on the projection of textual information. Lastly, in this research work, the use of frame markers is not suitable for the maintenance of bigger and more complex machines. Therefore, what is needed is more robust and more compact framework enhancing the technician’s mobility when working on the machine. In a more recent paper, a similar remote maintenance support system based on AR is presented [23]. It is remarkable that the authors have put a considerable amount of effort on providing a mobile tool, based on the utilization of tablet PC. However, similar to the previous work presented, the use of frame markers is inevitable, which can highly affect the overall performance of the tool and, by extension, the performance of the shop-floor/field technician.
Although there is a continuous development of the AR-based tools proposed by the research community during the last two decades, there is a lack of added functionality. The above-mentioned development as well as the practical implementation of these frameworks in real-life industrial scenarios and the in-vitro experimentation in laboratory environments has led to the conclusion that AR has to treated as a back-bone technology. By extension, in the near future, the development of AR tools has to be further enhanced by the addition of more functionalities. A representative example is the framework presented in Reference [24], where an adaptive AR framework for machine operation as well as technical maintenance is presented. The adaptivity of the framework is based on the skill level of the technicians. Similarly, in Reference [25], an innovative framework based on a wrist-haptic tracking methodology is introduced.

4. Maintenance under the Framework of Industry 4.0

The modern manufacturing world is undergoing a digitalization phase under the framework of Industry 4.0. During the current industrial revolution, digital technologies have tremendously advanced, which enables the improvement of other scientific fields, including that of technical maintenance. With the recent development in the ICT, and the integration of Internet of Things (IoT), the manufacturing domain has passed in a whole new phase by relying on the utilization of data, i.e., Big Data Analytics. For the successful completion of MRO besides the technical knowledge/expertise of the stuff carrying out the operations, it is crucial that all the needed components are acquired. Therefore, companies are maintaining an inventory of components. However, under the framework of Industry 4.0, and with the utilization of digital technologies, the existing inventory practices can be further improved through proper classification of the MRO components while additive manufacturing can facilitate in reducing lead times to produce several components [6][7][8][9][10][25][26][27].
For Industry 4.0 applications, two key objectives are to ensure maximum uptime throughout the production chain, and to increase productivity while reducing production costs. In Reference [28], a systematic approach to analyze the strengths and weaknesses of current open-source Big Data and stream processing technologies and assess their use for Industry 4.0 use cases is presented. In addition, the model of Industry 4.0 is gradually being implemented in worldwide development, distribution, and marketing chains. As such, the design and integration of a stainless-steel predictive maintenance system is presented in Reference [28]. Following the predictive maintenance field, the use of real-time detection and prediction algorithms regarding future failures has significantly benefited from the technological advances of Industry 4.0 era. There’s also growing interest in decision-making algorithms triggered by predictions of failure over the past few years. A literature review on decision making in predictive maintenance in the context of smart manufacturing is done in References [29][30]. Additionally, a review on predictive maintenance as an integrated predictive platform for production systems, focusing on maintenance approaches, methods, and tools is described in Reference [31].
Another significant issue is the estimation of the maintenance time for a new maintenance project, which is among the main maintenance offerings, is based solely on the experience and knowledge of the engineer. The research work presented in Reference [32] proposes a framework for knowledge-based calculation of maintenance time based on Key Performance Indicators (KPI) monitoring to support the capture and reuse of information in maintenance activities as well as to improve the performance of the given maintenance PSS. Next, an extendable and reusable scheduling approach in the context of predictive maintenance, which supports multiple heterogeneous inputs and outputs, is described in Reference [32]. In accordance with the current schedule, predictive maintenance indicators from the tracked equipment are used for scheduling maintenance operations. A web service architecture is implemented to accommodate highly different use scenarios such as suppliers of equipment.

This entry is adapted from the peer-reviewed paper 10.3390/app10051855

References

  1. Mourtzis, D.; Vlachou, E.; Milas, N.; Xanthopoulos, N. A Cloud-based Approach for Maintenance of Machine Tools and Equipment Based on Shop-floor Monitoring. In Proceedings of the 48th CIRP Conference on Manufacturing Systems, Ischia, Italy, 24–26 June 2015; pp. 655–660.
  2. Predictive Maintenance: Taking Pro-Active Measures Based on Advanced Data Analytics to Predict and Avoid Machine Failure. Available online: https://www2.deloitte.com/content/dam/Deloitte/de/Documents/deloitte-analytics/Deloitte_Predictive-Maintenance_PositionPaper.pdf (accessed on 2 January 2020).
  3. Wang, W.; Peter, W.T.; Lee, J. Remote machine maintenance system through Internet and mobile communication. Int. J. Adv. Manuf. Technol. 2007, 31, 783–789.
  4. Wang, J.; Feng, Y.; Zeng, C.; Li, S. An augmented reality based system for remote collaborative maintenance instruction of complex products. In Proceedings of the 2014 IEEE International Conference on Automation Science and Engineering (CASE), Taipei, Taiwan, 18–22 August 2014; pp. 309–314.
  5. Masoni, R.; Ferrise, F.; Bordegoni, M.; Gattullo, M.; Uva, A.E.; Fiorentino, M.; Carrabba, E.; Di Donato, M. Supporting remote maintenance in industry 4.0 through augmented reality. Procedia Manuf. 2017, 11, 1296–1302.
  6. Ong, S.K.; Zhu, J. A novel maintenance system for equipment serviceability improvement. CIRP Ann. 2013, 62, 39–42.
  7. Mourtzis, D.; Vlachou, A.; Zogopoulos, V. Cloud-based augmented reality remote maintenance through shop-floor monitoring: A product-service system approach. J. Manuf. Sci. Eng. 2017, 139, 061011.
  8. Mourtzis, D.; Zogopoulos, V.; Katagis, I.; Lagios, P. Augmented Reality based Visualization of CAM Instructions towards Industry 4.0 paradigm: A CNC Bending Machine case study. In Proceedings of the 28th CIRP Design Conference 2018, Nantes, France, 23–25 May 2018; pp. 368–373.
  9. Cardoso, S.F.L.; Mariano, Q.M.C.F.; Zorzal, R.E. A Survey of Industrial Augmented Reality. Comput. Ind. Eng. 2020, 139, 106159.
  10. Palmarini, R.; Erkoyuncu, A.J.; Roy, R.; Torabmostaedi, H. A systematic review of augmented reality applications in maintenance. Robot. Comput. Integr. Manuf. 2018, 49, 215–228.
  11. Fernández del Amo, I.; Erkoyuncu, A.J.; Roy, R.; Palmarini, R.; Onoufriou, D. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications. Comput. Ind. 2018, 103, 47–71.
  12. Berglund, F.Å.; Gong, L.; Li, D. Testing and validating Extended Reality (xR) technologies in manufacturing. In Proceedings of the 8th Swedish Production Symposium (SPS 2018), Stokholm, Sweeden, 16–18 May 2018; pp. 31–38.
  13. What Is Extended Reality Technology? A Simple Explanation for Anyone. Available online: https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/#69b6421f7249 (accessed on 16 February 2020).
  14. 4 Types of Maintenance Strategy, Which one to Choose? Available online: https://new.abb.com/medium-voltage/service/maintenance/feature-articles/4-types-of-maintenance-strategy-which-one-to-choose (accessed on 2 January 2020).
  15. Mourtzis, D.; Doukas, M. Design and Planning of Manufacturing Networks for Mass Customisation and Personalisation: Challenges and Outlook. In Proceedings of the 2nd CIRP Robust Manufacturing Conference (RoMac 2014), Bremen, Germany, 7–9 July 2014; pp. 1–13.
  16. Xun, X. From cloud computing to cloud manufacturing. Robot. Comput. Integr. Manuf. 2012, 28, 75–86.
  17. Mourtzis, D.; Vlachou, A. A cloud-based cyber-physical system for adaptive shop-floor scheduling and condition-based maintenance. J. Manuf. Syst. 2018, 47, 179–198.
  18. Azuma, T.R. A Survey of Augmented Reality. Presence Teleoperators Virtual Environ. 1997, 6, 355–385.
  19. Azuma, T.R. The Most Important Challenge Facing Augmenting reality. Presence Teleoperators Virtual Environ. 2017, 25, 234–238.
  20. Mourtzis, D.; Zogopoulos, B.; Vlachou, E. Augmented Reality Application to Support Remote Maintenance as a Service in the Robotics Industry. In Proceedings of the 50th CIRP conference on Manufacturing Systems, Taichung City, Taiwan, 3–5 May 2017; pp. 46–51.
  21. Henderson, S.; Feiner, S. Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Trans. Vis. Comput. Graph. 2011, 17, 1355–1368.
  22. Mourtzis, D.; Angelopoulos, J.; Boli, N. Maintenance assistance application of Engineering to Order manufacturing equipment: A Product Service System (PSS) approach. In Proceedings of the 16th IFAC Symposium on Information Control Problems in Manufacturing INCOM 2018, Bergamo, Italy, 11–13 June 2018; pp. 217–222.
  23. Mourtzis, D.; Xanthi, F.; Zogopoulos, V. An Adaptive Framework for Augmented Reality Instructions Considering Workforce Skill. In Proceedings of the 52nd CIRP Conference on Manufacturing Systems (CMS), Ljubljana, Slovenia, 12–14 June 2019; pp. 363–368.
  24. Siew, Y.C.; Ong, K.S.; Nee, C.Y.A. A practical augmented reality-assisted maintenance system framework for adaptive user support. Robot. Comput. Integr. Manuf. 2019, 59, 115–129.
  25. Chryssolouris, G. Manufacturing Systems: Theory and Practice, 2nd ed.; Springer: New York, NY, USA, 2006.
  26. Chen, J.; Gusikhin, O.; Finkenstaedt, W.; Liu, Y. Maintenance, Repair, and Operations Parts Inventory Management in the Era of Industry 4.0. In Proceedings of the 9th IFAC Conference on Manufacturing Modelling, Management and Control MIM 2019, Berlin, Germany, 28–30 August 2019; pp. 171–176.
  27. Bokrantz, J.; Skoogh, A.; Berlin, C.; Wuest, T.; Stahre, J. Smart Maintenance: An empirically grounded conceptualization. Int. J. Prod. Econ. 2019, 107534.
  28. Sahal, R.; Breslin, G.J.; Ali, I.M. Big data and stream processing platforms for Industry 4.0 requirements mapping for a predictive maintenance use case. J. Manuf. Syst. 2020, 54, 138–151.
  29. Ruiz-Sarmiento, J.R.; Monroy, J.; Moreno, F.A.; Galindo, C.; Bonelo, J.M.; Gonzalez-Jimenez, J. A predictive model for the maintenance of industrial machinery in the context of industry 4.0. Eng. Appl. Artif. Intell. 2020, 87, 103289.
  30. Bousdekis, A.; Lepenioti, K.; Apostolou, D.; Mentzas, G. Decision Making in Predictive Maintenance: Literature Review and Research Agenda for Industry 4.0. IFAC-PapersOnLine 2019, 52, 607–612.
  31. Efthymiou, K.; Papakostas, N.; Mourtzis, D.; Chryssolouris, G. On a Predictive Maintenance Platform for Production Systems. Procedia CIRP 2012, 3, 221–226.
  32. Nikolakis, N.; Papavasileiou, A.; Dimoulas, K.; Bourmpouchakis, K.; Makris, S. On a versatile scheduling concept of maintenance activities for increased availability of production resources. Procedia CIRP 2018, 78, 172–177.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
Academic Video Service
Feedback