IEEE Robotics & Automation Magazine - June 2020 - 43

manifold improved the detection of wrist movement for
most participants and proved to be efficient at locating
transitions between movements; see Figure 8 and Table 1
for a summary of the results. This shows the importance
and benefits of considering the underlying manifold structure of the data in this application. The details of this experiment can be found in [5].
Underwater Robot Teleoperation With
Task-Parameterized GMM
The fusion approach presented in the "Fusion With Products of Gaussians" section can be extended to the GMM
approach presented in the "Gaussian Mixture Model" section. This is particularly useful when mixtures of Gaussians are encoded in different coordinate systems that need
to be fused at reproduction time to satisfy constraints in
multiple frames of reference. Within the DexROV project
[41], this task-parameterized GMM (TP-GMM) approach
[4], [39] is used with the MPC approach presented in the
"Model Predictive Control" section to teleoperate an
underwater robot at a distance, with a teleoperator wearing
an exoskeleton and visualizing a copy of the robot workspace in a virtual environment.
Figure 9 presents an overview of this application (also see
[41] for a description of this teleoperation approach and [39]
for a general description of TP-GMM). Because of the long
communication delays between the teleoperator and the
robot, the locations of the objects and tools of interest are not
the same on the teleoperator and robot sides. Using a parameterization associated with the locations of objects and tools,
we can cope with this discrepancy by locally adapting the
movement representation to the position and orientation of
the objects/tools, represented as coordinate systems. Figure 9
depicts an example with two coordinate systems (with models represented in orange and purple) corresponding, respectively, to the robot and a valve that needs to be turned. A
motion relative to the valve and the robot is encoded as
GMMs in the two respective coordinate systems. During
teleoperation, each pair of GMMs is rotated and translated
according to the current situations on the teleoperator and
robot sides. Products of Gaussians are then computed at each
side to fuse these representations.

Table 1. The wrist movement detection results.
Rest

Wrist
Supination

Wrist
Extension

Wrist
Flexion

D
S ++

0.29 ± 0.00

0.18 ± 0.00

0.25 ± 0.00

0.27 ± 0.00

0.47 ± 0.00

0.31 ± 0.00

0.33 ± 0.00

0.32 ± 0.02

0.29 ± 0.14

0.36 ± 0.07

0.43 ± 0.13

0.46 ± 0.00

0.34 ± 0.00

0.35 ± 0.00

D
S ++

0.36 ± 0.02

0.22 ± 0.00

0.31 ± 0.00

0.29 ± 0.00

0.42 ± 0.00

0.43 ± 0.00

D
++

The table compares the root-mean-square error obtained by GMR on
the SPD manifold and the standard Euclidean GMR for wrist motion
estimation from sEMG (see [5] for details). The results are presented for
three participants.

Movements are encoded in this way with position R 3
and orientation S 3 data (in Figure 9, a representation with
R 2 and S 2 is shown as an illustration). For the position,
the retrieved path in black (and the associated covariances) correspond to a movement of the robot to the valve
(represented as red U shapes) by taking into account how
these different coordinate systems are oriented. We can
see that the approaching phase is perpendicular to the
coordinate system to properly reach the valve. For the orientation, the retrieved path shows how the orientation of
the end effector changes with time. At the beginning of
the motion, this orientation relates to that of the robot; at
the end, the orientation of the end effector matches that of
the valve.
In these graphs, the purple and orange ellipsoids depict
two GMMs representing uncertain trajectories with respect
to two different frames of reference (red U shapes for position data and red points on the spheres for orientation
data). The black ellipsoids indicate the final trajectory and
its uncertainty, obtained by fusing the trajectories of the
two different frames of reference through products of
Gaussians. Although the red U shapes and points are not
the same on the teleoperator and robot sides, the retrieved
paths on the two sides can quickly adapt to these different
situations. Using a Riemannian manifold framework,

Orientation Data
Robot Side

Orientation Data

Position Data

Teleoperator Side

Position Data

Figure 9. The TP-GMM extended to S d manifolds in the DexROV project.

JUNE 2020

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020