The OECD [2] has released the 2030 Future of Education and Skills report. It outlines the robot engineer’s job, which requires important skills due to the demand of technology for the future. It also emerged from an analysis of the industrial robotics market that robot software will be used in robot operations to achieve the specific objectives through the computer program coding [28]. It will exponentially grow between 2019 and 2025 due to the adoption of the Internet of Things (IoTs), Artificial Intelligence (AI), and other software technologies [29]. Many educational institutions are now adopting robotic programming as a part of their efforts to enhance students’ higher order thinking skills, beginning with the MIT Media Lab under the supervision of Professor Seymour Papert since 1985 [30]. To date, the use of robot programming processes has often been used in STEM (science, technology, engineering and mathematics) learning management [31,32,33,34].

In the article, the researchers investigate the two words “Robot” and “Programming” to determine with clarity the meaning of robot programming skills. We studied the meanings of both words by examining the meanings in scholarly dictionaries. We found that the Oxford Dictionary of Computing defines the term “robot” as “programmable devices consisting of mechanical actuators and sensory organs that are linked to a computer” [35], whereas the Oxford Dictionary of Computer Science defines the term “programming” as “all technical activities involved in the production of a program, including analysis of requirements and all stages of design and implementation. In a much narrower sense, it is the coding and testing of a program from some given design” [36].

From the meanings of the two terms presented above, it can be concluded that robot programming skills refer to “All technical activities related to the production of programs through the coding and testing of the program from the given design to the control of programmable devices consisting of actuators and sensors linked to the computer” [35,36]. This definition is consistent with the programming process synthesized by Lertyosbordin [15] with other academic sources [35,36,37,38,39,40,41,42,43]. The detail of the programming process consists of the following steps:

(1)

Identify the Problem: This refers to understanding the problem and determining the “Input”, “Process” and “Output” components that must be completed in order to solve the problem.

(2)

Design a Solution: This refers to the process of ordering the sequence of algorithms using flowcharts or pseudocodes.

(3)

Coding the Program: This is the way of transforming the commands and procedure sequence from the conceptual design into a programming language.

(4)

Test the Program: This refers to the validation of the syntax of the computer code and the interpretation of the results for the goals of program execution. It also includes testing for hardware compatibility, covering the input and output sections.

(5)

Program Implementation: This refers to the outcomes of the program. This should also be continued by further enhancements.

The researchers have defined the robot programming definition and synthesized the standard programming process [37,38,39,40,41,42,43]. We can then determine the components and indicators of the robot programming skill by applying the coding skills indicator of Surfing Scratcher [44], which was developed based on creating an educational measurement of Griffin [45], combined with the cognitive skills indicating verbs of Schraw and Robinson [46]. In addition, the researchers also analyzed the usage of verbs found in a variety of empirical studies [47,48,49,50,51,52,53] that evaluate cognitive skills in robot control programming tasks. The components and indications of the robot programming skill can then be synthesized as follows:

(1)

Component 1: The ability to solve problems step by step:

• Describe the problem and the sequence of ways to solve it.
• Draw the flowcharts or pseudocodes to show the sequence of ways to solve the problems.
• Change the sequence of steps if the results are not met.
• Tackle the presented tasks by breaking them down into smaller tasks.
• Capture the issues that can cause problems to repeat.

(2)

Component 2: The ability to create computer programs:

• Create a program by a computer language from a blank page.
• Create a program with a single-decision condition.
• Create a program with the nested structure of decision conditions.
• Create a variable to control the loop task programs.
• Create a variable and input data that affect the output.
• Build your own program from the beginning, until you achieve the objectives.
• Create a function that can modify parameters.

(3)

Component 3: The ability to connect to the robot:

• Connect the port between the computer and the microcontroller.
• Create objects for using analog and/or digital signals.
• Create a graphical user interface (GUI) to display the analog and/or digital inputs.
• Create a graphical user interface (GUI) for the digital outputs.

The concept of higher order thinking skills became an important educational topic when Bloom et al. [26] published the Taxonomy of Educational Objectives and described higher order thinking skills as Analysis, Synthesis and Evaluation. Later, in 1987, Resnick [54] researched the teachings of science and mathematics with an educational theme focusing on higher order thinking among public school students across the United States. The studies have shown that higher order thinking skills are important skills in the scientific thinking process and can be developed from the elementary school level and onward. In addition, Resnick said “Higher order thinking involves a cluster of elaborative mental activities requiring nuanced judgment and analysis of complex situations according to multiple criteria. Higher order thinking is effortful and depends on self-regulation” [54].

This is consistent with Lewis and Smith [55] who concluded that higher order thinking skills are the processes used to respond to situations through critical thinking and problem solving. Moreover, King et al. [56] stated that “Higher order thinking skills include critical, logical, reflective, metacognitive, and creative thinking. They are activated when individuals encounter unfamiliar problems, uncertainties, questions, or dilemmas” [49]. Later, in 2001 Anderson et al. [27] revised Bloom’s Taxonomy of Educational Objectives, pointing out that learners’ thinking characteristics should be divided into two dimensions consisting of the “Knowledge Dimension” and “Cognitive Process Dimension”. They have also modified Bloom et al.’s six stages of cognitive levels [26], but still define higher order thinking as starting at the analytic thinking stage, detailed in Table 1.

From the details of higher thinking skills mentioned above, it can be concluded that higher thinking skills refer to “the intellectual ability from the application of knowledge to the creation of new ideas of one’s own” [26,27,54,55,56]. In this research, we used the higher order thinking skills theory based on the revision of Bloom’s cognitive taxonomy of Anderson et al. [27], as the basis for designing the component and indicators in this study. The revision of Bloom’s cognitive taxonomy consists of the details about of “Knowledge Dimensions” and “Cognitive Process Dimensions”, which are as follows:

(1)

The knowledge dimensions:

• Factual—The fundamental understanding of terminology; scientific terms; labels; lexicon; slang; symbols or representations, and specifics, such as a knowledge of events, individuals, events, and information sources.
• Conceptual—Knowledge of a subject’s classifications and categories, concepts, theories, models, or frameworks.
• Procedural—Knowing how to perform a skill, procedure, technique, or methodology.
• Metacognitive—The method or approach of learning and thinking, being aware of one’s own cognition and being able to control, monitor and regulate one’s own cognitive process.

(2)

The cognitive process dimensions:

• Analyze—Breakdown a component and determine how the parts relate to one another and to an overall concept or purpose by differentiating, organizing, and attributing.
• Evaluate—Make decisions utilizing criteria and standards by checking and critiquing.
• Create—Integrate elements to create a coherent or functional whole; reorganize elements to create a new structure or pattern by generating, planning, and producing.

The higher order thinking assessment flourished in the 19th century to verify the validity of teaching methods for specific objectives and tried to determine the standard level of learners in each grade [57]. To date, the knowledge and cognitive process assessment has been used as part of building student enthusiasm and leading to the development of learners’ skills in accordance with the learning objectives [58]. Corliss and Linn [59] suggested a method for measuring thinking skills in scientific learning activities, presented in Table 2.

From Table 2, we found that higher thinking skills can arise in the scientific thinking process, where teachers can measure and assess students’ skills through learning activities. Therefore, in this study, the researchers used the higher order thinking skills dimension of Anderson et al. [27], which consists of the knowledge dimension and cognitive process dimension, presented in Table 3.

From Table 3, we can identify the higher order thinking by these 12 behavior indicators. In addition, in Computational Science, these behaviors refer to a group of competencies known as computational thinking. Selby [60] determines that the relationship of higher order thinking skills is directly linked to computational thinking, which consists of decomposition, abstraction, algorithm design, generalization, and evaluation. The relation is shown in Figure 1.

From Table 3 and Figure 1, it can be observed that we then have the higher order thinking indicators in the programming pedagogy. However, to assess the level of skill for each attribute, it is necessary to have a numerical rating scale to measure the performance. Leighton [61] shows that measurements can be made based on line 0–100 and divided into 5 levels (0–4), shown in Figure 2.

The rating scale was divided into five ranges, shown in Figure 2. This corresponds to Likert [62] that supports the design of the rating scale, which should be an odd number (3, 5, 7, …). If we consider Figure 2, it can be observed that the teachers should not assess learners by dividing them into only two sides (white and black) because some learners’ abilities are grayed out. Therefore, defining the middle of the scale is another suitable way to assess learners’ abilities more clearly.