IEEE Robotics & Automation Magazine - June 2020 - 64

such as power or precision but not both [26]. By leveraging
data-driven priors, the robot is not restricted to grasps similar
to those provided by the planner; instead, it can leverage any
grasp its learned model predicts will be successful. Indeed,
our resulting grasp planner reliably generates successful side
and overhead grasps on the real robot across several different
objects used for testing.
We can directly attribute this ability to generate both
overhead and side grasps to the new data set we generated
for this article, which contains substantially more successful
overhead grasps than our previous data set (Lu et al. [24]).
However, this improvement highlights that we have simply
shifted the burden of the external planner from inference
initialization to grasp exploration in generating training
data. While we overcame the bias of the planner somewhat
by adding random perturbations to the output of our heuristic planner, we still limited the space of grasps explored
during training.
To overcome this issue in the future, we wish to explore
active learning where the robot selects what grasps to attempt
for learning based on the previous attempted grasps and the
currently learned grasp model. This should improve the data
efficiency of our learning algorithm while also learning a
wider variety of grasps. However, new issues arise concerning
how to correctly update the learned model in an online fashion, as the standard independent identically distributed data
assumption used for batch NN training will no longer hold.
A more obvious shortcoming of our MDN prior stems from
the need to explicitly select the number of mixture components
in the model. An open question remains as to how we can
expand the capacity of the mixture network to encode a greater
variety of grasps as the robot collects more data for training.
As we noted in the section "Grasp Planning for Probabilistic Inference," our learned priors can be viewed as an approximation of the model uncertainty (i.e., epistemic [28]) of the
learned classifier. In future work, we wish to compare prior
learning with explicitly learning priors over the NN weights
w, which would hopefully provide better-calibrated predictions of the probability of grasp success. Accurate models of
the probability of grasp success would enable more reliable
task-level planning, where the robot could reason over the
probability of a sequence of events, producing the desired
outcome under uncertainty of the manipulated object's shape
and physical properties. However, it is unclear how one could
use such Bayesian NNs to efficiently perform MAP inference
for grasp planning, as a single evaluation of the NN uncertainty typically requires several forward-pass evaluations of
the NN model [28].
A final weakness of our results as presented stems from
our planner achieving better performance in attempted side
grasps than overhead grasps. This presents significant issues
in attempting to perform grasping in clutter or grasping of
low-profile objects where the hand must be close to the table.
Indeed, this problem arose in attempting to grasp the mug in
this article. We think learning or designing a more complex
feedback controller for overhead grasps using tactile feedback
64

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2020

would boost the overhead-grasp performance, especially for
objects with less contact areas on the top.
In conclusion, our article shows that we can improve
grasp planning as MAP inference by incorporating three
particular benefits. First, using a voxel-based object representation instead of an RGB-D improves learning performance.
Second, learning MDN priors represents an improvement
over uniform or object-independent learned priors. Three,
unsurprisingly, more data representing grasps of increased
variability improve grasp planning.
Nevertheless, several issues and open questions still
remain with our planning framework as presented. Clearly
learning-based approaches are becoming more and more
prevalent if not the norm for manipulation. We hope the
manipulation community takes hold of these questions and
finds more to further our understanding of grasp planning as
probabilistic inference.
Acknowledgments
Qingkai Lu and Balakumar Sundaralingam were supported in
part by National Science Foundation award 1846341.
References
[1] A. Saxena, J. Driemeyer, and A. Y. Ng, "Robotic grasping of novel
objects using vision," Int. J. Robot. Res., vol. 27, no. 2, pp. 157-173, 2008.
doi: 10.1177/0278364907087172.
[2] A. Saxena, L. L. S. Wong, and A. Y. Ng, "Learning grasp strategies with partial shape information," in Proc. AAAI Nat. Conf.
Artificial Intelligence, 2008, pp. 1491-1494. doi: 10.1609/aaai.v33i01.
33013542.
[3] I. Lenz, H. Lee, and A. Saxena, "Deep learning for detecting robotic
grasps," Int. J. Robot. Res., vol. 34, nos. 4-5, pp. 705-724, 2015. doi:
10.1177/0278364914549607.
[4] L. Pinto and A. Gupta, "Supersizing self-supervision: Learning to
grasp from 50K tries and 700 robot hours," in Proc. IEEE Int. Conf.
Robotics and Automation (ICRA), 2016, pp. 3406-3413. doi: 10.1109/
ICRA.2016.7487517.
[5] M. Kopicki, R. Detry, M. Adjigble, R. Stolkin, A. Leonardis, and J. L.
Wyatt, "One-shot learning and generation of dexterous grasps for
novel objects," Int. J. Robot. Res., vol. 35, no. 8, pp. 959-976, 2016. doi:
10.1177/0278364915594244.
[6] A. Mousavian, C. Eppner, and D. Fox, 6-DOF GraspNet: Variational
grasp generation for object manipulation. 2019. [Online]. Available:
arXiv:1905.10520
[7] B. Wu, I. Akinola, and P. K. Allen, Pixel-attentive policy gradient for
multi-fingered grasping in cluttered scenes. 2019. [Online]. Available:
arXiv:1903.03227
[8] M. Liu, Z. Pan, K. Xu, K. Ganguly, and D. Manocha, Generating
grasp poses for a high-DoF gripper using neural networks. 2019.
[Online]. Available: https://arxiv.org/abs/1903.00425
[9] A. Sahbani, S. El-Khoury, and P. Bidaud, "An overview of 3D object
grasp synthesis algorithms," Robot. Auton. Syst., vol. 60, no. 3, pp. 326-
336, 2012. doi: 10.1016/j.robot.2011.07.016.
[10] J. Bohg, A. Morales, T. Asfour, and D. Kragic, "Data-driven grasp
synthesis-a survey," IEEE Trans. Robot., vol. 30, no. 2, pp. 289-309, 2014.
doi: 10.1109/TRO.2013.2289018.

https://www.arxiv.org/abs/1903.00425

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020