IEEE Robotics & Automation Magazine - June 2020 - 63

multifingered grasping outperforms our previous RGB-D
image patch-based NN presented in Lu et al. [24] in terms of
both predictive accuracy and training time.
In our previous work, we showed that planning as inference
with the RGB-D structure outperforms several alternatives for
grasp planning, such as sampling and regression. As our results
here indicate improvement over the RGB-D network structure,
we can infer that our planning similarly outperforms alternative
multifingered grasp-planning approaches, while still running
fast enough for use in a deployed robotic system. Additionally,
our CNN classification approach to learning grasp success
allows for more data-efficient learning compared to directly
predicting grasps using regression. In the regression-based formulation, the NN takes the visual information (e.g., RGB-D or
voxel grid) as input and directly predicts a grasp configuration
as output. These regression models can learn only from successful grasps, while our success classifier learns from both successful and failed grasp attempts.
Using our learned, voxel-based classifier, we examined the
role of different prior probability distributions over grasp configuration in the planning process. Our real-robot grasp experiments for the three different prior models, defined in the
section "Grasp Planning as Probabilistic Inference," show that
using a learned object-conditional prior over grasp configurations benefits grasp inference when combined with the learned

grasp-success prediction network. This learned MDN benefits
from the same voxel encoder to represent the observed object
of interest. Furthermore, the data-driven GMM also provides
benefits over the bounded uniform prior. This provides further
evidence that learned priors provide better planning performance compared with heuristic, weak priors.
Learned priors provide additional benefits for multifingered grasp planning. First, they enable the robot to
directly sample initial configurations for use in the resulting optimization problem. In our previous work, we
showed how randomly generating numerous samples
from the uniform prior and selecting the one with highest
predicted success fails to reliably generate successful
grasps. This demonstrates the benefit of the learned prior,
which focuses the search space to promising configurations, something the uniform prior cannot provide. To
overcome this problem, we previously relied on an external grasp planner to initialize the optimization.
The second additional benefit of the learned prior comes
from its elimination of the need for an external planner or
heuristic for initialization. In addition to generally being computationally more efficient, the removal of this external planner reduces the bias present in human-designed planners,
which limit the space of grasps under consideration. Modelbased planners tend to prefer only a single class of grasps,

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

(p)

(q)

(r)

(s)

(t)

(u)

(v)

(w)

(x)

(y)

(z)

(aa)

(bb)

Figure 6. Examples of successful grasps generated by grasp inference using our voxel-based classifier with the MDN object-conditional
prior. (a)-(n) Side grasps and (o)-(bb) overhead grasps.

JUNE 2020

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

63
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
