IEEE Robotics & Automation Magazine - December 2018 - 28

"
"

Input: N(xt , Σt )
The estimated variability is
used to determine how
strongly the robot should
track the estimated path
with optimal control.
P
V
Output: Kt , Kt
P
..
x
K
xt
x
Kv
t
"

Reproduction
in New
Situation

∏j

What We Observe...
What We Learn...

Motion Observed
from Different
Perspectives
Frame 1
Frame 2

Reuse of
the Learned
Model in a
New Situation
by Exploiting
the Extracted
Task Variations

Multiple
Demonstrations

(a)

(b)

(c)

(d)

Only Gaussian ID
Is Transmitted
Figure 5. The shared control mode with two frames of reference
on each side. The current poses of the teleoperator and robot
are displayed with blue dots. ID: identification.

adaptation is weighted based on the variability of the
demonstrations in those parts of the task that are current-
ly executed. For parts of the task requiring accuracy, the
model assists the teleoperator by correcting deviations
from the original demonstrations. For parts of the task
allowing more variations, the teleoperator is free to move
within the regions corresponding to the demonstrations.
2) Semiautonomous control. The task is executed by gener-
ating the most likely trajectory starting from the current
pose of the robot. This mode is particularly useful when
delays or interruptions from the satellite communication
are expected. In this control mode, the teleoperator visu-
alizes and triggers the execution of the movement,
which continues until a new signal from the teleoperator
is given [5].
In the following, we summarize task learning and how
motions are reproduced in varying situations.
Learning Adaptive Tasks from Demonstrations
Our application requires the CE to learn skills based on only a
handful of demonstrations (typically up to ten). Additionally,
we require skills that can be reproduced in novel environmen-
tal situations, for which no demonstration is available. To
28

IEEE ROBOTICS & AUTOMATION MAGAZINE

december 2018

Robot Side

Teleoperator Side

Robot Side

Teleoperator Side

Figure 4. An overview of the learning approach used in the CE. (a) Demonstrations are collected with different task parameters
(frames of reference). (b) The demonstrations are transformed in each particular frame, and a GMM is learned in each frame. (c) In
a new situation, a new GMM is computed with a product of linearly transformed Gaussians. (d) The computed trajectory distribution
provides a variance estimate for each set-point, which determines how accurately the robot should pass through these set-points.

Figure 6. The semiautonomous control mode, with acceleration
commands and an associated trajectory computed from any of
the robot/teleoperator poses (displayed as blue points) using
the model.

achieve this goal, the CE relies on a task-parametrized Gauss-
ian mixture model (GMM) [6] to encode demonstrations exe-
cuted in different situations (Figure 4). The task parameters
are frames of reference (coordinated systems with position
and orientation information) associated to virtual landmarks
or objects/tools in the environment. For example, in a valve-
turning task, such frames may refer to the robot base frame,
the current valve pose, and the targeted valve pose.
Figure 4 depicts the learning and retrieval process. First,
demonstrations are collected in varying situations (each time
with different task parameters). To capture the variance of the
demonstrations, GMMs are learned in each task-relevant
frame. Learning the models in each individual frame allows
the system to generalize the observed tasks to new situations.
Task Reproduction with Adaptation
to New Situations
In a novel environmental situation, the Gaussian-mixture
components are transformed using the newly observed task
parameters [Figure 4(c)]. The retrieved GMM is exploited
differently according to the selected control mode, with the
aim of reducing the cognitive load on the teleoperator when
executing a set of tasks.

IEEE Robotics & Automation Magazine - December 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2018