Robot Arm Reaching Based on Inner Rehearsal

Robot Arm Reaching Based on Inner Rehearsal: Comparison

Please note this is a comparison between Version 1 by Gavin Wang and Version 4 by Fanny Huang.

Robot arm motion control is a fundamental aspect of robot capabilities, with arm reaching ability serving as the foundation for complex arm manipulation tasks. ThHowe researchers propose a robot arm motion control method based on inner rehearsal. Inspired by the cognitive mechanism of inner rehearsal observed in humans, this approach allows the robot to predict or evaluate the outcomes of motion commands before execution. By enhancing the learning efficiency of models and reducing excessive physical executions, the method aims to improve robot arm reaching across different platformver, traditional inverse kinematics-based methods for robot arm reaching struggle to cope with the increasing complexity and diversity of robot environments, as they heavily rely on the accuracy of physical models.

arm reaching
motion planning
inner rehearsal
internal model
human cognitive mechanism

1. Introduction

In recent years, robots have played important roles in many fields, especially for humanoid robots. As arm manipulation is one of the most basic abilities for human beings [1], arm motion control is also an indispensable ability for humanoid robots [2]. In modern factories, automation manufacturing and many other production activities are inseparable from robot arms [3]. Among various types of arm manipulation abilities, arm reaching is one of the most basic, and it is the first step in many complex arm motions, such as grasping and placing, and can also lay the foundation for subsequent motion, perception, and cognition ^[4][5][4,5].

The major goal of robot arm reaching is to choose a set of appropriate arm joint angles so that the end-effectors can reach the target position in Cartesian space with a certain posture, which is commonly referred to as internal model control (IMC) [6]. In 1998, Wolpert et al. [7] reviewed the necessity of such an internal model and the evidence in a neural loop. In the same year, Wolpert and Kawato [8] proposed a new architecture based on multiple pairs of inverse (controller) and forward (predictor) models, where the inverse and forward models are tightly coupled. Rresearchers usually implement robots’ motion control through kinematic or dynamic modeling [9]. However, since the control of force and torque is not involved in robot arm reaching tasks, only the kinematic method is considered. The internal kinematics model can be separated into the inverse kinematics (IK) model and forward kinematics (FK) model according to the input and output of the model.

In the task of reaching, the major problem is how to build an accurate IK model that maps a certain posture to a set of joint angles. There are mainly two types of approaches to robot arm reaching control: conventional IK-based approaches and learning-based approaches. Table 1 shows some related research.

Table 1. Classification of robot arm reaching control approaches.

Classification	Approaches
	Numerical method [10]
Conventional IK-based	Analytical method ^[11]^[12][11,12]
	Geometric method [13]
Learning-based	Supervised learning: deep neural networks [14], spiking neural networks [15] Unsupervised learning: self-organizing maps [16], reinforcement learning ^[17][18][17,18]

In conventional IK-based approaches, the mappin contributiong from the pose of the end-effector to the joint angles is built based on the mechanical structure of the robot’s arm. The joint angles are as followscalculated with an analytical method or iterative method [19].

The internal models are established based on the relative positioning method. Researchers limit the output of the inverse model to a small-scale displacement toward the target to smooth the reaching trajectory. The loss of the inverse model during training is defined as the distance in Cartesian space calculated by the forward model.

The models are pre-trained with an FK model and then fine-tuned in a real environment. The approach not only increases the learning efficiency of the internal models but also decreases the mechanical wear and tear of the robots.

The motion planning approach based on inner rehearsal improves the reaching performance via predictions of the motion command. During the whole reaching process, the planning procedure is divided into two stages, proprioception-based rough reaching planning and visual-feedback-based iterative adjustment planning.

2. Previous Studies

T In this typart describes previous studies on ve of approach, the accuracy of the IK model strongly depends on the measurement accuracy of the robot’s mechanical parameters, and researchers need to solve equations in high dimensions [20]. This bringsual servoing reaching, internal model establishment, and inner rehearsal difficulties in calibrating the parameters, which may change continuously because of the wear and tear of the robot. The application of conventional IK-based approaches in complex and unstructured environments is limited by the accuracy of the measurement and the manual calibration of the parameters.

To avoid the drawbacks of the conventional IK-based approaches, researchers implement the following.

Researchers use image-based visual servoing to construct a closed-loop control so that the reaching process can be more robust than that without visual information.

Researchers build refined internal models for robots using deep neural networks. After coarse IK-based models generate commands, researchers adjust the commands with learning-based models to eliminate the influence of potential measurement errors.

Inner rehearsal is applied before the commands are actually executed. The original commands are adjusted and then executed according to the result of inner rehearsal.

3. Overall Framework

Thishave focused on learning parmet describes the proposedhodologies to manipulate robot arm reachings, rather than control-theoretic approach in detaes, in recent years.

It il.s The overall framework, the establishment of the internal models, and the inner-rehearsal-based motionrecognized that robots must have the ability to learn continuously in order to adapt to the ever-changing and unstructured environment. Learning can also decrease the complexity of control for robots with large degrees of freedom [21]. The planningearning-based approaches are introduced.

Thespired by coverall framework of the proposed approach is shown in Figure 1gnitive, motion, and other relevant mechanisms in human beings. The pinveroposed method comprises four blocks: (1)se model is established by means of self-exploration, based on neural networks [22], vreisual information pnforcement learning [18], or ocessing, (2) target-drther learning algorithms.

Usiven planning, (3) inner rehearsal, and (4) command execution.

Figure 1. The overall framework of the robot arm reaching approach based on inner rehearsal. The purple shading in some boxes denotes “inner rehearsal”, different from “visual information processing”. The use of cylinders and rectangles is intended to represent different types of components in the system. Cylinders indicate “models”, while rectangles indicate “values”.

The target position in Cartesian space is generated after the robot sees the target object through the visual perception module. The visual stimulation is converted into the required visual information, and then the intrinsic motivation is stimulated to generate the target

¹⁹

The aim of movement is generated by the relative position between the target and the end-effector. The inverse model generates the motion command based on the current arm state and the expected movement. Each movement is supposed to be a small-scale displacement of the end-effector toward the target.

The forward model will predict the result of the motion command without actual execution. The predictions of the current movement are considered to be the next state of the robot so that the robot can generate the next motion command accordingly. In this way, a sequence of motion commands will be generated. The robot conducts (2) and (3) repeatedly until the prediction of movements exactly reflects the target.

The robot executes these commands and reaches the target.

4. Experiments

Tg a learning-based model, accurate measurements of the robot’s mechanical parameters are no evaluate the effectiveness of the proposedonger the decisive factor in arm reaching approach based on inner rehearsal, several. However, a well-performing model requires a large amount of training data, mainly generated from trial-and-error experiments are conducted. Some ex, which might cause great abrasion to the robot. Many researchers train the model in simulation first and then refine the model in a real robot platform [23].

To periformental settings and analyses well in robot arm reaching tasks, it is necessary to ensure the accuracy of the results are introduced in this section. In the visual part, because it is not the key research part,internal model and the target positioning accuracy as well. As discussed above, in arm reaching manipulation, the joint angles can be calculated by the inverse kinematics model in IK-based approaches or the inverse model built through learning-based approaches once researchers simply process the image in the HSV and depth s know the position and posture of the end-effector, while the target pose is mainly determined through visual positioning, which is strongly related to the performance of the camera.

Compacre to extract the target information. It is encoded as a Cartesian (x,y,z)d with the absolute positioning method, relative positioning will help to reduce the influence of perception error. Research also shows that in the process of human arm reaching, older children and adults consider both target and hand positions [24,25].

T Based on this me chanism, Luo et al. [26] proposed a relative-location-based approximation approach is verified on the Baxter robot and the humanoid robot PKU-HR6.0 II. To confirm the position of the endn 2018. In their work, the reaching process is divided into two stages, rough reaching and iterative adjustment. However, the motion commands are combined with six basic moving directions, which may lead to non-smooth reaching trajectories. To smooth the trajectory of the robot armeaching trajectory, researchers add a red mark to the end-effector and the robot detects its position throughout the experiment.

5. Conclusions

A use differential theory for reference and limit the distance of each movement to a small-scale given threshobotld.

The arm reaching approach based on inner rehearsal is proposedes described in the above research are mainly in an open-loop manner [27,28]. TEache internal models are pre-trained with a coarse FK model and fine-tuned in joint of the robot arm moves to a target angle calculated by the inverse model, where feedbacks are less considered. However, when it comes to the real environment. The two-stage le, the planning becomes more difficult and less robust [29], arning ofd the motion commands generated by the internalverse models helps to improve the learning efficiency and reduce may not be executed precisely because of mechanical wear and tear. Theerrors. Thus, feedbacks on the execution and corresponding motion planning approach based on inner rehearsal improvesre needed to increase the reaching performance by. In 2017, Krivic et al. [30] proposed a planning approach to predict new ing the result of a mformation about a partially known state and to decrease uncertainty. In 2019, Luo et al. [31] proposed an action command with the forward model. Based on tselection method based on sensorimotor prediction and anticipating the consequences of currently executable actions in internal simulation.

2. Reaching with Visual Servoing

The generelative distance, the al aim of the visual servo system in reaching tasks is to reduce the error

e (t)

[33], defined as

e ( t ) = s ( m ( t ) , a ) − s ∗

wholere

m (t)

replresents image meansurements, a inding procecates any potential additional data, and

s^{*}

stores is divided into propriothe desired final position.

As an effective meption-based rough reachingthod, visual servoing has been applied in motion estimation [34], position control [35], and other robotics tasks. Ining and 2000, Shen et al. proved that with visual-feedback-based iterativ servoing, the robot trajectory can approach the desired trajectory asymptotically [35]. In 2013, Lampe et adjustment planning, which improves thel. proposed a reinforcement learning system for autonomous robot arm reaching performance. The experimental results show that researcherusing visual servoing. They used a visual feedback control loop to realize control, making it both reactive and robust to noise [17].

3. Learning-Based Internal Model

In 1987, Kos'uge method improves the effectivenesset al. introduced the concept of a virtual internal model and applied it in robot arm rcontrol [36]. In 1993, Koivisto et al. aching tasks. For the operation problem of the robotic arm, researchers' work nalyzed a nonlinear internal model controller realized with a multilayer perceptron neural network and proved that implements a relatively fixed two-stage framework. The human cing practical experience with neural networks can provide IMC robust performance [37], demognitive mechanism has great potential in enablistrating that a learning-based internal model is reliable.

Ing agents torecent years, the learn to determine the sing-based internal model has been used in ethical robots [38], robot intent prategic fediction [39], ramework of graspingbot manipulation [18], byand themselves, so thatother robotics research; it simulates the cognitive mechanisms of humans and makes robots more intelligent.

4. Inner Rehearsal

Humans can bsimulate the more suitablnormal execution results of an action through inner rehearsal [40]. Taking advantage ofor unknown, complex scenes. Furthermore, researchers can delve deepe the inner rehearsal mechanism, people can try to run the potential actions in their minds and predict their results, such as actions or decisions that are not explicitly executed [41,42].

For inroboto the cognitive mechanisms of humans and investigate how these insightss, through inner rehearsal, the result of a motion command can be predicted without actual execution [32]. caInn further enhance er rehearsal has been used in robot learning andinguistic interpretation [43], drecision making.lation learning [44], Ian doingd navigation [45] sto, researchers can unlock new possibilities for predict the result of a command and choose an action accordingly. In this way, robotic capabilities can avoid unnecessary attempts and their seamless integration into various applicationsconduct the best move. In their recent work, Atkinson et al. proposed to use pseudo-rehearsal in reinforcement learning to prevent neural networks from catastrophic forgetting [46,47].

FK	Forward Kinematics
IK	Inverse Kinematics
IMC	Internal Model Controller
DIM	Divided Inverse Model
FM	Forward Kinematics Model
IM	Inverse Kinematics Model
DoF	Degree of Freedom