Encyclopedia
Over the past decade, the use of AR has significantly increased over a wide range of applications. Although there are many good examples of AR technology being used in engineering, retail, and for entertainment, the technology has not been widely adopted for teaching in university engineering departments. It is generally accepted that the use of AR can complement the students’ learning experience by improving engagement and by helping to visualise complex engineering physics; however, several key challenges still have to be addressed to fully integrate the use of AR into a broader engineering curriculum.
- augmented reality
- AR
- engineering education
1. Introduction
XR, also known as extended reality, refers to a combination of real and virtual environments where interaction between humans and machines is established through computer-generated technology and compatible hardware [1]. Currently, the most common “X” representations are virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR creates a digital environment where the user is wholly immersed in a virtual world. AR overlays (augments) digital content into the user’s real-world environment. MR aims to blend both virtual and real-world environments where both coexist and interact with each other [2]. In engineering, the use of AR has been a key part of the industry 4.0 concept that focuses on advanced technologies in manufacturing systems and factories [3][7]. The use of AR has been utilised in many ways by industry including visualising complex assemblies, facilitating training programmes, and developing maintenance programmes and manufacturing lines [4][8]. In engineering education, the use of AR is significantly increasing compared to other STEM subjects [5][12]. AR is being used as part of course material (i.e., [6][7][13,14]), to visualise engineering laboratories (i.e., [8][9][15,16]), in collaborative projects [10][17], for communication skills such as industry presentations [11][18], and to enhance students’ spatial cognition [12][13][19,20]. In the past ten years, several forms of AR have been studied: using handheld (i.e., [14][15][21,22]), handsfree (i.e., [16][17][23,24]), and special hardware kits [18][19][25,26]. In addition, AR can be integrated with several digital technologies such as gamification [20][21][27,28], IOT [22][23][29,30], machine learning [24][31], FEA [25][32], and CFD [6][13] to produce more realistic, interactive, and scientifically credible AR experiences.
2. AR in Engineering Education Applications
Figure 1 shows the engineering fields where AR has been implemented in an engineering curriculum. Most of the uses were in mechanical, electrical, and civil engineering. The use of AR to date can be divided into three categories: first, delivering principles of engineering in selected topics within the curriculum, for example, HVAC systems [26][27][50,51], hydraulic transmission [28][52], Unified Modelling Language (UML) [29][53], hydraulics laboratory [22][29], CAD design [13][30][31][32][33][34][20,41,54,55,56,57], manual material handling (MMH) laboratories [16][17][23,24], oscilloscope training [35][36][58,59], or circuit analysis [21][24][28,31]. Such studies focus on developing AR experiences and then finding the impact the experience has on the student’s understanding. AR also allows users to observe an internal structure that cannot be observed in standard laboratory environments such as the simulation of electron movements [37][60] or the behaviour inside a nuclear core that students would usually have to imagine from limited observable data [7][14]. Secondly, AR experiences can be used to visually enhance the engineering learning material and make the laboratory components and some equipment more portable and accessible [38][39][40][41][71,88,89,90]. These studies tend to focus on enhancing the teaching material and evaluating the students’ interest and motivation towards the lectures. Thirdly, AR can be used to display digital instructions to physical spaces to teach students how to complete a task using hands-free mixed-reality AR devices [16][17][42][43][23,24,66,91].
3. Tool and Technology
For software, the options for developing AR experiences in an engineering context can be divided into three categories: (i) the use of mobile application development and 3D modelling tools [44] (such as Unity, Vuforia, Arkit, or ARcore), (ii) the use of in-house development tools, and (iii) the use of already developed AR experiences [41]. For hardware, the use of AR experiences can also be divided into three categories: (i) handheld smart devices, (ii) head mounted devices [16][17][26][29][42], and (iii) special hardware kits [7][18][19][45][46][47][48][49][50]. A summary of the software and hardware technologies is shown in
For software, the options for developing AR experiences in an engineering context can be divided into three categories: (i) the use of mobile application development and 3D modelling tools [99] (such as Unity, Vuforia, Arkit, or ARcore), (ii) the use of in-house development tools, and (iii) the use of already developed AR experiences [90]. For hardware, the use of AR experiences can also be divided into three categories: (i) handheld smart devices, (ii) head mounted devices [23,24,50,53,66], and (iii) special hardware kits [14,25,26,77,78,81,82,83,100]. A summary of the software and hardware technologies used in this review is shown in
Figure 2
.
Figure 2. Tools and technology used for AR in engineering education.
Using image targets is the common way to develop AR in engineering education experiences. These augmentations either use a barcode to launch the AR experience or are in the form of AR textbooks. This is due to how easy it is to use and support image markers with all commercial AR SDKs. The use of surface markers and holograms is less common compared to image markers and, in this context, was mostly used with HMDs or special hardware kits. Although the use of AR mainly focuses on visually enhancing the student learning experience, recently, the use of AR is being associated with another industry 4.0 technology, the internet of things (IOT) which being used for AR in engineering education [6][22][23][42][51].
Using image targets is the common way to develop AR in engineering education experiences. These augmentations either use a barcode to launch the AR experience or are in the form of AR textbooks. This is due to how easy it is to use and support image markers with all commercial AR SDKs. The use of surface markers and holograms is less common compared to image markers and, in this context, was mostly used with HMDs or special hardware kits. Although the use of AR mainly focuses on visually enhancing the student learning experience, recently, the use of AR is being associated with another industry 4.0 technology, the internet of things (IOT) which being used for AR in engineering education [13,29,30,66,69].
The hardware technology used to experience AR content has significantly improved over the past decade. Experiences constructed between 2010 and 2012 [8][50] required several trackers, an HMD, a personal computer (PC), and a camera to view an image marker experience. Today, users can view high-quality image marker experiences only using their smart devices. While the use of handheld smart devices is most common among AR education applications, the use of hands-free AR is becoming more popular as the Microsoft HoloLens 2 availability is increasing. The use of AR HMDs has shown great benefits in terms of safety, cost, and engineering training. In the aviation industry, for example, the use of HMDs can save manufacturers significant amounts of money and provide an updated safe training experience. However, in engineering education, the use of AR HMDs can still be considered expensive especially if more than one headset is needed for each laboratory session. The augmented reality sandbox [18][19][52] is an example of using specialist hardware to generate a mixed reality experience. The sandbox hardware [18][19][52] consists of a computer with a high-end graphics card running Linux, a Microsoft Kinect 3D camera, a digital video projector with a digital video interface, and a sandbox with a Kinect camera mounted above the box.
The hardware technology used to experience AR content has significantly improved over the past decade. Experiences constructed between 2010 and 2012 [15,100] required several trackers, an HMD, a personal computer (PC), and a camera to view an image marker experience. Today, users can view high-quality image marker experiences only using their smart devices. While the use of handheld smart devices is most common among AR education applications, the use of hands-free AR is becoming more popular as the Microsoft HoloLens 2 availability is increasing. The use of AR HMDs has shown great benefits in terms of safety, cost, and engineering training. In the aviation industry, for example, the use of HMDs can save manufacturers significant amounts of money and provide an updated safe training experience. However, in engineering education, the use of AR HMDs can still be considered expensive especially if more than one headset is needed for each laboratory session. The augmented reality sandbox [25,26,67] is an example of using specialist hardware to generate a mixed reality experience. The sandbox hardware [25,26,67] consists of a computer with a high-end graphics card running Linux, a Microsoft Kinect 3D camera, a digital video projector with a digital video interface, and a sandbox with a Kinect camera mounted above the box.
4. Users' Feedback
Feedback from students is a key factor to assess the success of the implementation of AR experiences as they are the primary users of the proposed technology. The students’ feedback has been assessed from different perspectives. Generally, survey questions can be categorised as follows:
- Motivation and interest: how did the AR material affect the student motivation toward the presented material?
- Learning material: is the presented material suitable for AR and does it improve the student’s understanding?
- Ease of use for the whole AR experience in classroom and remotely.
- Educational added value.
- Overall experience, and positive/negative attitude toward the use of the technology.
In addition to the previously listed categories, particular attention to other aspects of the AR experience was given attention by some AR researchers. These survey elements could significantly enhance the overall evaluation methods for AR-based technologies. These categories are as follows:
5. Strength, Weakness, Opportunity and Threat (SWOT) Analysis
- AR technology uniquely provides students the ability to observe internal structure, complex engineering physics (such as fluid flows, heat distributions, currents, and magnetic fields), guidance to complete hands-on tasks, and link real-world applications with taught material in a safe interactive environment.
- Affordable software and hardware that can be used to develop and consume AR experiences are increasingly available.
- The ‘WOW factor’ associated with the use of these technologies encourages student engagement.
Weaknesses
- There is a lack of AR digital assets for engineering principals developed by educators.
- There has been very little integration of AR experiences into engineering curricula.
- There have been very few studies on the long-term educational impact on both students and educators.
- ]
Opportunities
- Student achievement could be improved because of better engagement, motivation, enhanced visualisations, and improving the students overall learning experiences.
- AR technology could be a vehicle for other industry 4.0 concepts to be included in education.
- AR offers a means to customise the learning experience for students based on their capabilities and their learning preferences.
- [
Threats
- Lack of skills to develop AR experiences from engineering students and educators.
- Limited commercialisation of developed AR engineering applications.
- Educators not adopting the AR applications, due to the lack of AR digital assets.
- 50].
- Software solutions: installation and running [8][15], third-party providers [41][90], application stability [53][61].
- Digital Assets: feedback on the produced AR media [8][30][41][54][15,41,90,108].
- Hardware used: hands-free, hand-held, and eco-system.
- Comparative analysis between hands-on and AR lab [45][77].
The AR literature reports only limited feedback from educators and most of the educators’ feedback was recorded from a student’s perspective when using the technology and not from an educator’s point of view. There remain many unknowns when it comes to the use of AR technology in education including “How much time does it take a lecturer to develop an AR experience [43]?”, “What previous knowledge is required to develop an AR experience?”, “How much testing, debugging, and optimisation does an AR experience require”, and “How does the use of AR affect lecturers’ work and cognitive loads?”.
The AR literature reports only limited feedback from educators and most of the educators’ feedback was recorded from a student’s perspective when using the technology and not from an educator’s point of view. There remain many unknowns when it comes to the use of AR technology in education including “How much time does it take a lecturer to develop an AR experience [91]?”, “What previous knowledge is required to develop an AR experience?”, “How much testing, debugging, and optimisation does an AR experience require”, and “How does the use of AR affect lecturers’ work and cognitive loads?”.
