IEEE Robotics & Automation Magazine - June 2020 - 23

movements. As a replacement for manual robot program- trajectories, resulting from two different models, to allow the
ming, learning from human demonstrations, also called imi- obstacle to be passed from the left or right.
tation learning, has been proven a promising and powerful
In many applications, human demonstrations might use
technique for intuitive robot programming [1]. Inspired by multiple modes and models at the same time. To avoid both
neuropsychological findings [2], we use MPs as essential mode and model collapse, we propose learning a mapping
building blocks for describing robot motions. In this context, from the task parameter query q to a GMM of the MP
the question of how a generalizable and compact representa- parameters w with MDNs. Each mixture component of the
tion of MPs can be learned from demonstrations and adapted GMM represents either a mode for one specific task parameto new situations and changing task parameters is an active ter query or a model for multiple task parameter queries.
research area in robotics.
However, training an MDN with only the negative log-likeliDifferent MP representations have been proposed, such as hood (NLL) cost, as is usually done in the original MDN,
dynamic MP (DMP) [3], probabilistic MP (ProMP) [4], and might still lead to mode and model collapse, especially when
task-parameterized GMM (TP-GMM) [5]. These representa- the set of demonstrations is relatively small, as shown in Figtions can adapt to task parameters in the space in which we ure 1. To further reduce the occurrence of both collapses, we
define these primitives. For example, a DMP adapts to a new introduce an entropy cost function for training the MDN
start or goal; a ProMP adapts to intermediate via points; and (entropy MDN).
TP-GMM adapts to changes of predefined local frames,
Figure 1(a) shows the human demonstrations (dashed
where we describe the demonstrated trajectories. However, curves) for obstacle avoidance with an imbalanced distribufor different tasks, the parameters could have different tion of examples associated with the two different modes as
meanings and are not directly associated with spatial or well as the results obtained with different approaches for a
temporal requirements for motion trajectories. For exam- new task parameter query (the green dot). As can be seen, our
ple, a robot throwing a ball to a target specified in the Car- approach (entropy MDN) generates multiple solutions (here,
tesian space by executing a motion learned and represented five solutions are shown in Figure 1) that cover both demonin the robot's joint space should be able to adapt the learned stration modes (blue and green) for the same task parameter
motion to new targets. None of these approaches can adapt query. We achieve this result with the entropy cost function,
a learned MP to new targets specified in a different space.
which ensures a more balanced probability associated with
For such generalization problems,
methods have been proposed in
which task-specific mapping be Multiple Modes
Multiple Models
tween the task and MP parameters is
learned through regression models,
such as locally weighted regression
Demonstrations
(LWR) [6], [7], Gaussian process
regression (GPR) [8], or deep neural
networks [9]. However, such methMode Collapse
Model Collapse
ods cannot deal with the problems of
mode and model collapse. Given the
task of reaching a target while avoidGPR
ing an obstacle (see Figure 1), the
goal can be reached using two different modes, i.e., by trajectories passing
the obstacle from the left or right.
Thus, learning an MP for such a task
Original MDN
should take multiple modes in
human demonstrations into account
to increase the diversity of motions,
which is beneficial in the case of
changing task constraints.
Entropy MDN
Even though there are no multiple
modes for each individual task parameter query in human demonstrations,
there might exist multiple models for
(a)
(b)
different types of task parameters. As
shown in Figure 1(b), the goals on the
Figure 1. The (a) mode and (b) model collapse issues in the obstacle-avoidance problem
left and right sides of the obstacle are are solved using the entropy MDNs. The dashed curves denote the demonstrations, and
reached separately by two types of the solid curves show the generated trajectories.
JUNE 2020

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020