IEEE Robotics & Automation Magazine - June 2020 - 26

share a common network part (the blue box), which allows
extracting common latent features.
We assume that the covariance is a diagonal matrix R.
According to [17], a GMM with a diagonal variance matrix
approximates any given density function to arbitrary accuracy. The diagonal covariance output has the same dimension as
the mean output. For K components, an MDN has K pairs of
mean and variance outputs. Each pair corresponds to a mixture
component of a GMM.
The dimension of the MP parameters, as the output of the
MDN, determines the accuracy of the trajectory representation. With more SEKs, the MP represents the motion in a
more accurate way. In the following experiments, we use
10 SEKs for each dimension. As an example, the output mean
vector has 30 dimensions for a 3D motion trajectory, and there
are a total of K + K # 30 # 2 values in the MDN output.
For one single demonstration, we calculate w by solving
the least-square problem for (2). With M demonstrations,
a training data set " (q i, w i) ,iM= 1 is collected. The NLL is
written as
M

K

i =1

k =1

l NLL (H) = - / log e / r k (q i; H) N (w i; n (q i; H), R (q i; H)) o,
(5)

where H is the parameters of the network, and
N ^w i; n i, R ih =

(2r )

1

d/ 2

1/2

|R i |

· exp e - 1
2

d

(w i, j - n i, j) 2

j=1

v i, j

/

2

o,

(6)

where d is the dimension of the output, n i = n (q i; H), and
R i = R (q i; H). A stochastic gradient descent method is used
to minimize the NLL.
According to previous works [18], [19], training an MDN
with the NLL suffers from mode collapse. In [18], to avoid
the mode collapse and reduce learning complexity, Hjorth
and Nabney suggest fixing the mean and variance on a grid
defined in the output space and training only the model that
outputs the mixing coefficients to reduce the NLL. If there
are enough components regularly distributed in the output
space, fixed means and variances do not reduce the representation capability. However, for large-dimensional outputs,
such as MP parameters, this method leads to a sizeable
intractable grid.
In [19], Makansi et al. used an MDN to predict the distribution of future car positions based on a current car
position. To avoid mode collapse, they separated the MDN
into two parts: a sampling and inference network. The sampling network takes the current car position as input and
outputs a fixed number of hypotheses for future car positions. The authors trained the sampling network to place
hypotheses to cover all of the observed outputs diversely.
Based on these hypotheses, an inference network infers the
parameters of the GMM. The MDN is a combination of the
sampling and inference networks. The proposed method
avoids mode collapse for the car-position prediction.
26

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

JUNE 2020

However, for a high-dimensional output, such as MP
parameters, the sampling network requires a large output
dimension. Hence, it is difficult to apply both methods to
our problem.
Entropy Costs for the MDN
Before introducing the entropy cost function, we first inspect
the reasons that mode and model collapses occur when learning an MDN from demonstrations. Mode collapse occurs if
the demonstrations associated with different modes for a task
parameter query are imbalanced. For example, in Figure 1(a),
only a small number of demonstrations take the path from
the right side. By maximizing the likelihood of all demonstrations, the MDN tends to output a small mixing coefficient for
the mixture component, which corresponds to the mode with
fewer associated training data. In theory, it is correct to associate a small probability with an event that rarely happens in the
observations. However, the reasons for the imbalance of the
demonstrations in different modes, such as the habit of the
demonstrator, can be meaningless for correct motion generation. It is often the case that we cannot collect enough demonstrations to cover all modes. Even if there are only a few
demonstrations, where the human accomplishes the task with
particular types of motions, the robot should learn these
motions to increase motion diversity.
Model collapse occurs in particular when relatively few
demonstrations are available. Several mixture components of
the MDN, which are represented by neural networks, are powerful enough to overfit all demonstrations. After training of the
MDN, instead of all K mixture components, it uses only a subset of them, which corresponds to the local minima of the
NLL and results in poor performance of the MDN for some
task parameter queries. As shown in Figure 1(b), one of the
two models disappears with the original MDN, and the MDN
performs similarly to the GPR. Compared to mode collapse,
model collapse is more severe because it can lead to the failure
of task execution.
To reduce the occurrence of mode and model collapses,
we introduce the negative model entropy cost function
as follows:
l model (H) =

K

/ p (m = k | D ; H) log p(m = k | D ; H),

k=1

(7)

where
p (m = k | D ; H) =

M

/ r k (q i; H) p(q i),

i=1

(8)

where m is the component index, and p (q i) ? M -1. By minimizing the cost, we increase the uncertainty of the model
labels when considering all demonstrations D. A high uncertainty of the model labels is equivalent to either equally distributed mixing coefficients for each task parameter query or
equally distributed demonstrations to different models. In the
former case, if all mixing coefficients for one specific task
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
