IEEE Robotics & Automation Magazine - June 2023 - 65

quickly or even beyond it. The performance
of the proposed algorithm is compared
with the BC algorithm where the
agent is trained with supervised learning
and optimized by Adam, and the 64
batch size is specified. Besides, GAIL is
sensitive to the parameter ;m hence, the
values 0.1, 0.01, and 0.001 of m were
tested to find the best value. Figure 8
depicts the results showing that the modified
GAIL algorithm outperforms other
algorithms in terms of both convergence
speed and agent performance and that the
model performs optimally with m taken
as 0.01. Besides, for simple tasks, the proposed
algorithm can outperform manual operations, while for
complex tasks, the proposed algorithm can achieve levels
close to those of manual operations. The details of the experiments
in the real robot are elaborated in the next section.
"
FOR DRL ALGORITHMS,
WE COMPARED THE
CURRENT MAINSTREAM
ALGORITHMS, SUCH AS
THE PPO, A3C, AND SAC
ALGORITHMS.
„
EVALUATION OF THE REAL-WORLD ASSEMBLY
In the real robot experiments, the neural network parameters
are first trained in a simulated environment and then transferred
to the real world. The experiments were conducted on a
Baxter robot equipped with two arms with seven degrees of
freedom. To evaluate the performance of
the robot using RL methods, a total of
three comparison baselines were set.
They are as follows:
1) DM: The experimenters grab the
wrists of the arms by hand and move
them to complete the assembly in
zero gravity mode. This situation is
chosen as the best-case baseline for
the task. The experimenters are able
to see the workpiece directly, thus
enabling the completion of the gripping
and assembly. An ideal algorithm
should control the robot arm as
fast and accurately as DM.
2) Monitor and keyboard (MK): The experimenters used a
27-in computer monitor with 4K resolution to view the live
scene and a keyboard interface to control the robot's endeffector
translations and rotations via RViz software [19].
3) Scripted agent (SA): A scripting agent, which is written by
professionals, acquires the position of the workpiece and
the states of the robot arms and automates the control.
Different algorithmic choices are ablated in Table 3. Removing
the Transformer structure significantly reduces the success
rate of the assembly, which proves that the Transformer
TABLE 3. Ablations of the components of the DRL algorithm.
SUCCESS RATE
METHOD
No Transformer
No image augmentation
One historical state
Five historical states
Ten historical states
BC
DOUBLET A
0.69 ± 0.08
0.84 ± 0.1
0.79 ± 0.08
0.84 ± 0.06
0.83 ± 0.06
0.72 ± 0.04
DOUBLET B
0.54 ± 0.07
0.71 ± 0.11
0.61 ± 0.07
0.73 ± 0.09
0.73 ± 0.08
0.61 ± 0.08
DOUBLET C
0.38 ± 0.1
0.59 ± 0.14
0.52 ± 0.09
0.60 ± 0.11
0.61 ± 0.1
0.49 ± 0.09
AVERAGE
0.537
0.713
0.64
0.723
0.723
0.607
Agent
0 s
Script
10 s
20 s
30 s
40 s
FIGURE 9. Agent versus script operators in the assembly task. The images of episodes executed by the agent (top) and script (bottom)
are shown every 10 s. The agent completes the assembly task faster than the script operator.
JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
65

IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023