IEEE Robotics & Automation Magazine - June 2020 - 24

the different modes. The original MDN generates multiple
solutions, but these solutions (also five) reflect only one mode
in the human demonstrations since it is associated with a high
probability for the dominant mode. The GPR generates only
one single solution that is close to the examples for the dominant mode.
Figure 1(b) shows the demonstrations (blue and green
dashed curves) associated with two different models and
the trajectories generated by different approaches (solid
curves) for several task parameter queries (the colored
dots). As shown at the bottom of Figure(b), our approach
(entropy MDN) successfully learns the two models and generates the solution for the task parameter query correspondingly. We also achieve this result with the entropy cost
function, which ensures that each model is associated with
some human demonstrations. The original MDN has a higher chance of being stuck in the local minima of the NLL,
where only one model is trained and overfitted for all task
parameter queries. This fact leads to the failure of the task
execution, i.e., collision with the obstacle, especially when the
task parameter queries are on the boundary of two models.
The GPR can learn only one model; thus, it suffers from the
same problem as the original MDN.
For many tasks, it is often easier to collect failed samples with
an underfitted MDN model instead of requesting additional
human demonstrations. To further improve the MDN performance, we introduce a failure cost function that reduces the
occurrence of the same failures for a given task parameter query.
Related Works
DMP is one of the popular approaches to representing robot
motions. A DMP consists of a damped spring system and a
nonlinear force term f (x) such that
xvo = K ( g
xyo

- y) - Dv + ( g - y 0) f (x) x,

= v,
N

f (x) =

/ } i(x) w i

i=1
N

/ } i(x)

(1)

,

MP Generalization

i=1

where v and y are the scaled velocity and position of the trajectory point, respectively; g, y 0, and x, as the hyperparameters, represent the goal, start, and temporal factor separately;
and x is the canonical variable, which goes from 1 to 0. The
force term f (x) is a linear regression model with N squared
exponential kernels (SEKs) } i (x) = exp (- h i (x - c i)2),
where h i , and c i are fixed constants. w = (w 1, f, w N )T is a
vector representing the DMP parameters.
DMP generalization means replacing w with a parameterized function ~ (q). As mentioned previously, this function
can be represented and learned with different approaches,
such as GPR [8], LWR [6], [7], support-vector regression
(SVR) [10], or deep neural networks [9]. To train those models, the training data set is collected as M pairs of task parameter queries and MP parameters " (q i, w i) ,iM= 1, where the ith
24

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

MP parameter w i is learned from the ith demonstration and
corresponds to the ith task parameter q i .
Those methods require two parameterized functions, f (x)
and ~ (q). The output of ~ (q) is the parameter w of f (x);
hence, they are called two-step methods. In [11], Stulp et al.
proposed combining two functions into one single function
f (x, q), which was learned with LWR or GPR and extended
to the GMM in [12]. Since these methods use only one single
function, they are called one-step methods, and they are more
compact than the two-step methods.
The idea of the TP-GMM, suggested in [5], is to observe
human demonstrations from multiple perspectives (local
frames). A global GMM represents the trajectory points in a
global frame and maximizes the likelihood of demonstrations from different perspectives. With the transformation of
the local frames, the TP-GMM achieves a better extrapolation performance than other methods. In [13], Pignat and
Calinon extended the TP-GMM and learned the sensory
data together with the motion trajectories. As mentioned
before, however, the task parameters considered by the TPGMM are limited.
The ProMP uses a linear regression model with kernel
functions }($) to directly represent the motion trajectory
y (x) = } (x)T w. In [4], Parachos et al. assumed that w follows
a Gaussian distribution. In [14], the Gaussian distribution was
extended to a GMM. For human-robot interactions, the
ProMP parameter ^w = " w o, w c ,h is separated to encode
both human (w o) and robot motions (w c). With the conditional probability, the robot MP parameter w c is inferred
based on the human MP parameter w o . Both methods in [13]
and [14] learn a generative model and use the conditional
probability to infer unknown variables based on known ones.
In this article, we use a via-points MP (VMP), an MP formulation presented in our previous work [15] and described
in the "The VMP" section. Instead of learning a generative
model, we learn an MDN to directly map from a task parameter q to a GMM of the MP parameters w. As a GPR or SVR
for DMP, our technique belongs to the two-step methods.

*

JUNE 2020

The VMP
We use the VMP (see [15]) to represent robot motions, which
consists of an elementary trajectory h and shape modulation f
as follows:
y (x) = h(x) + f (x) = g + x ( y 0 - g) + } (x)T w,

(2)

where x is the canonical variable, which goes from 1 to 0 with
a linear-decay canonical system. The elementary trajectory
takes the form of a polynomial; here, it is a first-order polynomial, i.e., a line connecting the start and the goal. By changing
y 0 and g as the hyperparameters, the VMP adapts to the new
start and goal. As the nonlinear force term in DMP, the shape
modulation is a linear regression model with the SEKs }. w is
the parameter vector of the VMP.
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
