1000/1000
Hot
Most Recent
Augmented reality (AR): overlays virtual objects on the real-world environment and also provides the ability to interact with that environment. AR is any case in which an otherwise real environment is “augmented” by means of virtual (computer graphic) objects. With the continuously growing usage of collaborative robots in industry, the need for achieving a seamless human–robot interaction has also increased, considering that it is a key factor towards reaching a more flexible, effective, and efficient production line. As a prominent and prospective tool to support the human operator to understand and interact with robots, Augmented Reality (AR) has been employed in numerous human–robot collaborative and cooperative industrial applications.
This analysis was used to answer the three research questions: (Q1) What are the main AR visualization technologies used in industrial Human--Robot collaboration and cooperation context? (Q2) What are the main field of application of AR in industrial Human--Robot collaboration and cooperation context? and (Q3) What is the current state of the art of AR applications for Human--Robot collaboration and cooperation? Is research focusing on experimental or concept applications? What are the most used assessment techniques and indicators? What are the Research gaps presented in AR for industrial Human--Robot collaboration and cooperation context?
The first major contribution of this paper is the identification of the main AR visualization technologies used in industrial human--robot collaboration and cooperation context (research question Q1). HMD interfaces have gained relevance since 2018, followed by a not as much exponential interest growth of projected interfaces. On the other hand, Fixed Screen and HHD interfaces have fallen into disuse for HRC applications in the past three years. As mentioned by De Pace [4], these results are not unexpected: HMD and projector-based interfaces are hands-free, which enable greater operator mobility within the industrial warehouse and does not require as much attention shift as Fixed Screen and HHD approaches.
It is acceptable to assume that these characteristics would allure researchers to focus new studies on these two interfaces, especially on HMD. Moreover, since newer headsets are capable of performing scene understanding and natural gesture interactions without the need of external devices, it is expected that the research interest of this devices quickly grows. The same argument is valid when comparing the Augmented Reality glasses. Neither the Epson MOVERIO[53] nor the Acesight S[54] do not provide the same capabilities as the Microsoft HoloLens[55], which explains the researchers' preference for using the Microsoft device. Therefore, the exponential growth of studies involving HMDs can also be related to the first version of the Microsoft HoloLens in 2016.
Comparing the projector types used for Human--Robot collaboration, there is not a clear preference towards a particular projection method. Even though brightness, resolution, contrast, and other factors must be taken into consideration when choosing a projector, the selected papers lead to a faint tendency towards LCD and LED approaches.
Although all HHDs were based on Android, it is not possible to assume this operating system as a preference due to the small sample size. Some factors that may justify the inclination of researchers towards Android are the potentially lower cost, the larger number of platforms and devices, the easier adoption for developers, and the APIs to take full advantage of the embedded sensors.
Concerning tracking methods, there is a tendency to use markerless approaches. That is mostly driven by projection systems, which use feature extraction software for tracking object positions. However, marker-based approaches are gaining strength, especially due to the HMDs. The straightforward implementation of marker-based approaches and the lower computational power required might be the reasons that sustain this increasing tendency. On the other hand, the new Augmented Reality HMDs have a reasonable computational power and numerous sensors and cameras for spatial mapping, which may favor the use of markerless approaches over time.
The second major contribution of this paper is the identification of the main fields of application of AR in industrial human--robot collaboration and cooperation context (research question Q2). The use of augmented reality for improving the operator's awareness, trust, and safety feeling towards the robot is a constant concern, making the safety category the most researched one. Since the operator needs to share the workspace with one or more robots, any additional layer of safety may help improve the operator's condition and feeling towards the robot and the collaborative scenario. Although no Augmented reality safety approaches are yet certified, some of them have been showing promising results as active monitoring systems.
The capacity of augmented reality to display real-time information to the user field of view is essential to insert the human in the loop. Providing production live status, graphics, simulations, analysis and other operation-related information in a fast and intuitive way enables the user to be more assertive when making a decision that can affect the whole production line, or responding to emergencies.
Moreover, by showing to the user step-by-step text, images, videos, 2D or 3D representation of a workpiece, or animated instructions, augmented reality is vastly used for guidance applications. The main contributions for guidance applications are improving the operators' performance by reducing the task completion time and abstracting the user of the necessity of memorizing numerous instructions for a variety of products, lowering their cognitive load, and enabling an untrained operator to be able to conduct any operation if needed. Augmented reality is also being vastly used for programming applications. Most of the applications developed in this field fall under the Human--Robot Interaction category since no further collaboration happens among the parts after programming. Finally, quality control is an area that is starting to be combined with augmented reality and is expected to gain more relevance over time.
Regarding the economic activity, the automotive industry conducts the largest number of studies by a considerable amount. This may be because the automotive industry has always been a pioneer when comes to testing emerging technologies and especially to the fact that it presents numerous challenges that are a perfect fit for evaluating different applications, from a smaller to a larger scale. Lastly, again by a large difference, assembly operations are the most studied activity when it comes to augmented reality applied for human--robot collaboration and cooperation.
The third major contribution of this paper consists of the identification of the current state of the art on AR applications for Human--Robot collaboration and cooperation (research question Q3). It is possible to infer that qualitative evaluations are currently more employed than quantitative evaluations. It indicates some uncertainty about the usage of Augmented Reality wearable equipment, especially on human--centered perspectives, such as usability and ergonomics terms. The most used evaluation techniques are the System Usability Scale (SUS) questionnaire, for evaluating the system usability, and the NASA-Task Load Index (NASA-TLX) questionnaire, for assessing the participants' cognitive load. On quantitative evaluations, the most used measurement is the task execution time, for assessing performance improvements, and the Rapid Upper Limb Assessment (RULA) technique, for evaluating ergonomic aspects.
AR applications demonstrated the ability to improve operators' performance, accuracy, task understanding, safety feeling, and task awareness when compared to baseline assistance methods, such as printed and fixed screen based instructions. Moreover, the change of purely manual tasks to an AR-assisted collaborative scenario was demonstrated to reduce the production cycle time and improve the operator's ergonomics. Concerning visualization, HMDs are deferred due to hardware aspects such as narrow field of view, occlusion, and weight, which might exert some influence on the operator's safety feeling.
Industries will soon employ AR applications for a variety of cooperative and collaborative tasks. Therefore, future developments should rely on end-user experience and feedback to further improve human--robot collaboration features. Moreover, multi-agent communication and interaction between different operators, devices, and services should be explored to promote more dynamic information exchange throughout the industry. Thus, there is still much development and research to be done and questions to be answered before understanding and achieving the full potential of Augmented Reality on Human--Robot Collaboration and Cooperation.