IEEE Robotics & Automation Magazine - December 2018 - 79

Another approach for trajectory optimization is model
predictive control (MPC) [15]. MPC seeks the optimal trajectory from the current state to the target state over a finite time
horizon, and the first optimal input is applied to the system.
Although its computation time can be adjusted by modulating the length of the time horizon, the convergence domain
becomes drastically contracted if the system is highly nonlinear or has many constraints. Additionally, the performance of
MPC depends significantly on the initial guess because
underlying optimization algorithms used in every step usually
give the local optimum.
In short, optimization-based motion planners may well
not compute the globally optimal solution; thus, they are subject to the initial guess. In addition, they cannot provide a
solution in a short time.
Sampling-Based Motion Planning
Rather than solving difficult problems in motion planning,
sampling-based techniques use random computations. They
exploit random sampling and build a set of waypoints, providing a fast solution even for high-dimensional and constrained problems. Moreover, because we need only the
criterion to decide suitable waypoints, the algorithm is
straightforward, and the application is relatively easy.
A rapidly exploring random tree star (RRT*), an extended
form of the RRT [16], uses simple optimizing processes that
connect or reconnect the waypoints as they improve the cost.
By repeating those processes in every iteration, the algorithm
guarantees asymptotic optimality.
For the cooperative aerial task, we can use an RRT* as a
planning algorithm by considering the dynamics of the aerial
manipulator in the local planning of the RRT* [5], [13]. An
RRT* asymptotically finds the optimal pose of each aerial
manipulator along the trajectory, which allocates proper payload. By simply checking whether the sampled states meet the
conditions, an RRT * handles the multiple constraints inherent in the interaction between agents.
To achieve a close-to-optimal solution, an RRT* often
demands a much longer computational time than that
required to obtain the initial solution. Recent research regarding both optimization- and sampling-based motion planning
has improved the rate of convergence by using smart sampling or developing a simple formulation of the problem. Still,
neither optimization- nor sampling-based planning alone is
satisfactory for cooperative aerial tasks that require a fast solution to react appropriately to sudden risky situations.
Learning-Based Motion Planning
Learning Representation of Motion
Motion-representation algorithms have been used to reproduce a given trajectory robustly. Dynamic movement primitives (DMPs) are powerful motion-representation techniques
because, as model-free approaches, they demand a low computational load during task execution. In DMPs, we calculate
the forcing term to follow the demonstration in given DMP

equations. Using supervised learning to form the forcing
term, the robotic system can quickly modify the trajectory so
that the original demonstration can be robustly followed by
perturbed configurations.
As described in the previous section, we emphasize the
importance of motion optimality and quick reaction for aerial
cooperation. The successful combination of an RRT* and
DMPs can address the inherent tradeoff issues of each. Moreover, the robustness of DMPs is advantageous in the presence
of perturbations such as the internal forces between cooperating agents or their out-of-sync movements.
Although each individual demonstration in a specific environment can be reproduced using DMPs, it is not sufficient as
a motion planner for generalization in different environments.
So how can we generalize DMPs? If we can reveal the relationship between the surroundings and the corresponding optimal
motion, the optimal motion may be inferred from new information about the actual environment. This type of relationship
can be obtained from multiple demonstrations; to set the
DMPs' framework for multiple demonstrations, parametric
skills may be employed. The parametric-DMPs (PDMPs) presented in [17] adapt style-variable modeling to DMPs.
The quality of the represented motion depends on the demonstration. To generate optimal demonstrations, an RRT* is
used. In particular, an RRT* is advantageous in obtaining distinctive features because it is less prone to local minima. Thus,
the combination of RRT*s and PDMPs enables fast and efficient motion planning as PDMPs allow fast computing and
RRT*s provide efficient trajectories to learn. As mentioned previously, in the process of optimizing the RRT*, the common
payload is distributed automatically to each aerial manipulator.
RRT*-PDMPs
The detailed process of the framework for PDMPs using
RRT* demonstrations, i.e., RRT*-PDMPs, is described in
[13]. Figure 3 provides an overview of the proposed learningbased planning algorithm.
First, the equation of DMPs [18] is represented as
p
a3 q

a 1 (g -

q) -

a 2 a 3 qo - a 1 (g -

q 0) | +

a 1 f (|; w),

(1)

where q denotes the state vector of the system, and g and q 0
are the goal and start configuration, respectively. In (1), |
and f (|; w) are introduced to make q asymptotically converge to the unique point g. Here, | is the phase variable
with dynamics described as |o = - a 4 | in [19], which forms
a canonical system. a 4 is the constant value, | = 1 indicates
the start of the time evolution, and | close to zero means
that the goal g has essentially been achieved. f (|; w) is an
attractor function, which we previously called the forcing
term. The attractor function f forces the current-state variable to the demonstrated trajectories. By learning the weights
w, we can form the attractor function, which robustly represents the target attractor function computed from the demonstrated trajectories. a 1, a 2, and a 3 are the time constants,
december 2018

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - December 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2018