IEEE Robotics & Automation Magazine - September 2022 - 72

only a fraction of them can handle deformation. One recent
work, Softgym [52], was proposed for benchmarking DOM
based on NVIDIA Flex. In the soft robotics community,
SOFA [53] and ChainQueen [54] are example simulators. In
the sections " Gripper and Robot Design " and then " Challenges
and Outlook, " we considered the interaction between soft
robots and deformable objects. Thus, a unified simulator that
is able to handle soft robots and objects and model their interaction
might be desirable.
When choosing a model for control, one challenge of datadriven
deformation modeling is to balance the region of
validity with the amount of data required for training. One
possible direction is to combine a simple model with a complex
nonlinear model to form a hierarchical model. An example
of such structures is exploited in [55] for robust in-hand
manipulation. For DOM tasks, we can have a linear model at
the lower level and a (D)NN learning the full model at a higher
level. The lower-level model can be learned in a few iterations
to enable instant interaction between the robot and
object. The higher-level (D)NN can collect data and improve
the model to enhance global convergence.
Planning
Current Capability
Planning aims at finding a sequence of valid (robot/object) configurations
and contributes to solving the problem of limited
validity of local models, as discussed in the section " Modeling. "
Planners can operate in the objects' configuration space
and sometimes rely heavily on physics-based simulation.
While the obtained plan can be visually plausible, it may be
unrealizable for a specific object. Recently, McConachie
et al. presented a framework that combines global planning
without physics simulation, with local control [56]. For an
elastic object, considering its energy is another way to do
planning; in this direction, Ramirez-Alpizar et al. [57] proposed
a dual-arm manipulation planner optimizing the elastic
energy, for elastic ring-shaped object manipulation. For
DOM tasks involving multiple robots, planning is important
for coordination. Alonso-Mora et al. employed a distributed
receding horizon planner for transporting tasks that require
multiple robots [58]. More recently, the work of [59] learned
a latent representation for semantic soft object manipulation
that enables (quasi)-shape planning with deformable objects.
With learning from demonstration (LfD), the robot can be
trained to manipulate deformable objects by an expert (usually
a human). LfD encodes the robot trajectory and interaction
force from human demonstrations [60], thus avoiding explicitly
planning the motion. More recently, Wu et al. have proposed
a reinforcement learning scheme for DOM that does
not require initial demonstrations [61].
Challenges and Outlook
A rigid object configuration can be described in space with
6 DoF, whereas a deformable object configuration has a
much higher number of DoF. To address this from the
72 * IEEE ROBOTICS & AUTOMATION MAGAZINE * SEPTEMBER 2022
sensing algorithm side, one can find a compact representation
from sensory data, as discussed in the section " Challenges
and Outlook " under the section " Sensing. " An
alternative that receives much less attention is the use of
environmental contacts to constrain some DoF of deformable
objects. Examples include the use of contact points in
cable harnesses or that of flat surfaces when folding clothes.
We argue that, instead of planning to avoid contacts-as
most planners do-for deformable objects, we need to plan
to make contact since this constrains the configuration and
therefore simplifies the task.
Planning to grasp the correct point is often crucial in
DOM tasks. For instance, grasping at convex vertices of the
clothes guarantees stability and facilitates the task [62].
Regrasp planning is highly relevant when considering tasks
that require multiple robotic arms. Additional challenges
come from perception since as soon as the robot releases one
or more grasp(s), the object is likely to change its configuration.
We rely on sensing to track configuration changes and
then plan accordingly.
Another important future work in planning is reasoning
about a deformable object at a semantic level. What does it
mean for a cloth to be folded? What does it mean for an
object to be wrapped in a paper? We cannot manually specify
all of the configurations of the deformable object to use as
goals in these kinds of tasks. Instead, we need a way to learn
the meaning of semantic concepts, such as folded or wrapped,
so that we can determine if a given configuration of the object
is a valid goal.
Control
Current Capability
Control aims at designing inputs for the robot to realize the
planned motion. The type of controller is usually decided by
the task. For instance, the authors employed a data-driven
model predictive control [63] for cutting considering its predictive
nature and the lower demand for manual tuning. For
safe interaction in minimally invasive surgery, the authors of
[64] used a fuzzy compensator with impedance control. For
controlling large deformations, Aranda et al. proposed a
shape-from-template algorithm concerning its low-dimensional
representation (using the template) and robustness
against occlusion [65].
Several works focus on shape control. While global models
directly map sensor data to robot motion, local models
must be inverted to design the robot motion controller (see
the section " Modeling " ). Several applications of the control
scheme for robotic manipulation of deformable objects can
be found in computer, communication, and consumer manufacturing
[66], [67], where vision-based controllers were
proposed to drive the robot to automatically grasp/contact a
deformable object and then carry out the task of active
deformation or separation/sorting. Other works consider
the concept of diminishing rigidity to perform deformation
control [68], [69].
IEEE Robotics & Automation Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2022
