Aesthetics captures the user expectations of the game. In
[16][68], eight aesthetics properties are defined, including sensation, fantasy, narrative, challenge, fellowship, discovery, expression, and submission. A game does not need to satisfy all eight aesthetics properties. Game mechanics are mostly defined by the rules of the game, which are inevitably game specific. Some common game content may also be considered as part of game mechanics, such as points, levels, and leader boards
[15][67]. Game dynamics refers to how the game is implemented in a system that would give the player aesthetic experiences. Although game dynamics are game specific, there are some common elements that help achieve game aesthetics, as shown in
Figure 35. Feedback is an essential component of game dynamics. Games typically incorporate some form of virtual reality, which helps achieve user sensation and fantasy. Incorporating time pressure during game play and having an opponent would increase the challenge of the game. Supporting multiple players to cooperate during game play would create fellowship experience. Allowing customization of the game environment, such as allowing the user to select an avatar or even create a unique avatar, would help satisfy the players’ expression desire.
4.2. Virtual Reality
Virtual reality refers to the immersive and interactive environment of a synthetic world
[17][72]. The synthetic world might incorporate elements of the real environment to different extents. In
[17][72], this is referred to as a virtuality continuum. For example, the virtual reality environment may use smartphone cameras and sensors to incorporate the user’s face or movement into the virtual reality scene. That is why some publications use the term “augmented” reality.
Traditionally, virtual reality is achieved via expensive 3D immersive systems such as CAVE and head-mounted goggles. Recently, devices have been made to use a smartphone in a way similar to head-mounted goggles. The Virtuoso mobile app is used in this way
[18][30], and it was used to train adults with ASD adaptive skills. However, a virtual reality user interface does not have to be fully immersive to be effective, which has been confirmed in many autism studies
[19][11].
4.3. Autism-Specific Mechanisms
Several studies incorporated autism-specific mechanisms to improve the engagement level of the mobile apps. In
[20][17], the app design allows the user to choose an avatar that matches their own restricted interest. As a result of the alignment with the child’s restricted interest, the mobile app (called Sidekick!) was able to maintain the motivation of the child with ASD and improve the treatment adherence in therapy sessions, schools, and at home.
5. Guidelines Followed in the Mobile App Development
Three studies provided detailed processes in the design and development of the mobile app
[21][22][23][16,21,31]. All of them followed a user-centered design (UCD) process
[24][74]. UCD refers to a loosely defined process that has the following elements: (1) understand the users; (2) actively involve the users when defining the requirements; and (3) iterative design and evaluation.
To understand the users and to discover the requirements, various instruments have been used, including questionnaires, interviews, focus groups, and observations. Ideally, the targeted individuals with ASD should be the subjects of these instruments. However, it might not be practical to directly involve children with ASD in the design process due to their disorders. Some of them might be non-verbal. As such, studies often involve parents, teachers, and therapists of the children with ASD. Experienced software developers for this type of app have also been involved
[21][22][16,21].
Each of the three studies developed or followed a set of guidelines to make the app a good fit for the target autism population and for the specific application objective, as illustrated in
Figure 46. The development of the PuzzleWalk app
[23][31], which is an app promoting physical activities for adults with ASD, included ten guidelines as shown in the left column of
Figure 46. The development of the SimpleTEA app
[21][16], which is a serious mobile game for improving communication in children with ASD, considered four major perspectives: user sensitivity, user safety, user engagement, and game playability, as shown in the middle column of
Figure 46. The development of the PlanTEA app
[22][21], which is an app supporting children with ASD to attend medical appointments, was guided by the AutismGuideline proposed in
[25][75], as illustrated in the right column of
Figure 46.
Figure 46. Specific design considerations for the autism population in three studies.
6. The Theoretical Foundations in the Mobile App Design to Ensure Intervention Effectiveness
6.1. Augmentative and Alternative Communication
Due to the tendency of social aversion and difficulty in communication in individuals with ASD, AAC has been pervasively used in autism intervention programs. AAC refers to the methodology of helping individuals with ASD to overcome the difficulty in communication. AAC helps achieve five goals of social interaction [26], as shown in Specific design considerations for the autism population in three studies.
6. The Theoretical Foundations in the Mobile App Design to Ensure Intervention Effectiveness
6.1. Augmentative and Alternative Communication
Due to the tendency of social aversion and difficulty in communication in individuals with ASD, AAC has been pervasively used in autism intervention programs. AAC refers to the methodology of helping individuals with ASD to overcome the difficulty in communication. AAC helps achieve five goals of social interaction [76], as shown in 7:
Figure 57. AAC goals in social interaction (
left) and AAC techniques (
right).
-
Communication of needs and wants. This is the most basic form of communication. It often involves the individual making a request to the caregiver and expecting an action-oriented response.
-
Information transfer. This form of communication entails the formulation of a coherent message. Hence, it is more complex and demanding for the individual. The ability of making this form of communication is essential for an individual to receive education, stay employed, and receive healthcare.
-
-
Social closeness. To establish friendship and other inter-personal relationships, sustained conversational communication is required. This is an even more demanding form of communication. In addition to coherent information, this form of communication requires the individual to express one’s emotions and to have a correct understanding of the feelings of the conversation partner.
-
Social etiquette. This form of communication is necessary for an individual to conform to social conventions of politeness, for example, the individual is supposed to express appreciation when being helped.
-
Internal dialogue. This form of communication is to communicate with oneself, which is necessary to accept oneself, to be productive daily, to perform self-reflection, and to improve oneself.
-
AAC techniques can be divided into unaided and aided communication [27], as shown in
AAC techniques can be divided into unaided and aided communication [77], as shown in Figure 5. Unaided communication is also referred to as total communication. In total communication, the individual would augment speech (if any) with manual signs and gestures. In aided communication, the individual would use physical objects, pictures, drawings, or technology to augment speech or as an alternative to speech. A PECS is one of the most popular aided communication techniques.
6.2. Video Modeling
Video modeling is considered a form of evidence-based practice in autism intervention [28]. Video modeling refers to using video clips to demonstrate how to complete a task or the desirable behavior in a social interaction. Video self-modeling has been used in many studies, where the learner could be video-recorded while performing a task and simulating a scenario [28]. Typically, a therapist would view the recording and provide feedback. Smartphones and tablets constitute an excellent platform for an individual to acquire various skills and to gain knowledge on proper ways of social interaction via video modeling and video self-modeling. The concept of learning via modeling is attributed to Bandura, who elaborated the role of modeling in social learning [29]. As a method for autism intervention, video modeling has been used to improve social communication skills, functional skills (for daily living), and behavioral functioning [28].
6.3. Behavior Changes
7. Unaided communication is also referred to as total communication. In total communication, the individual would augment speech (if any) with manual signs and gestures. In aided communication, the individual would use physical objects, pictures, drawings, or technology to augment speech or as an alternative to speech. A PECS is one of the most popular aided communication techniques.
6.2. Video Modeling
Video modeling is considered a form of evidence-based practice in autism intervention [78]. Video modeling refers to using video clips to demonstrate how to complete a task or the desirable behavior in a social interaction. Video self-modeling has been used in many studies, where the learner could be video-recorded while performing a task and simulating a scenario [78]. Typically, a therapist would view the recording and provide feedback. Smartphones and tablets constitute an excellent platform for an individual to acquire various skills and to gain knowledge on proper ways of social interaction via video modeling and video self-modeling. The concept of learning via modeling is attributed to Bandura, who elaborated the role of modeling in social learning [79]. As a method for autism intervention, video modeling has been used to improve social communication skills, functional skills (for daily living), and behavioral functioning [78].
6.3. Behavior Changes
6.3.1. Behavior Change Techniques
A large number of behavior change techniques (BCTs) have been developed
[30][80]. A BCT includes one or more mechanisms that induce behavior changes, and typically follows a number of components as illustrated in
Figure 68 [31][60]. The key steps of a BCT include goal setting, action planning, behavioral practice, and ultimately habit formation. The PuzzleWalk design was guided by a BCT
[23][32][31,33] to promote a more physically active life style.
Figure 68.
Main steps in behavior change techniques.
6.3.2. Cognitive Behavioral Training
Cognitive behavioral therapy (CBT) is well established in addressing psychological issues. CBT has been successfully used to reduce anxiety and improve social skills in children with ASD
[33][62]. CBT aims to identify the cognitive conditions that result in undesirable behaviors and make corresponding improvements at the cognitive level. Generally, CBT consists of six components, as illustrated in
Figure 79.
Figure 79. Main components in cognitive behavioral training.
7.3.3. Behavior Modeling Training
Behavior modeling training (BMT) is another framework for behavioral changes [34][35]. The most distinctive characteristic of BMT is modeling, which is derived from social learning theory [29]. Both positive-only models and mixed modes have been proposed and evaluated for behavioral changes [34]. The main steps in BMT are illustrated in Main components in cognitive behavioral training.
7.3.3. Behavior Modeling Training
Behavior modeling training (BMT) is another framework for behavioral changes [63,81]. The most distinctive characteristic of BMT is modeling, which is derived from social learning theory [79]. Both positive-only models and mixed modes have been proposed and evaluated for behavioral changes [63]. The main steps in BMT are illustrated in Figure 8.
Figure 8. Main steps in behavior modeling training.