IEEE Robotics & Automation Magazine - June 2020 - 60

(a)

(b)
Figure 2. The mean visualizations of two MDN mixture components for five different objects. (a) Mean of the overhead component.
(b) Mean of the side component.

Table 1. Number of successful and failed side and
overhead grasps in the simulation-based training
and offline-testing sets generated using the
heuristic planner.
Training

Offline Testing

Grasp Type

Success

Failure

Success

Failure

Side

1,559

2,616

339

515

Overhead

749

4,064

157

810

[24]. In that work, overhead grasps were randomly selected to
align with either the major or minor axis of the bounding box
top face, but this fails to generate overhead grasps robustly. In
this article, we improve the overhead heuristic planner by
aligning the thumb to point in the direction of the mean vector related to the major and minor axes of the bounding box
top face, with the palm parallel to the top face. This modification boosts the overhead grasp success rates in data collection.
We generated a preshape by randomly sampling joint angles
for the first two joints of all fingers within a reasonable range,
fixing the last two joints of each finger to be zero. There are 14
parameters for the Allegro hand preshape, six for the palm
pose, and eight relating to the first two joint angles of each finger proximal to the palm. Given a desired pose and preshape,
we use the rapidly exploring random tree-connect motion
planner in MoveIt! to plan a path for the arm. We executed all
feasible plans moving the robot to the sampled preshape.
After moving the hand to the desired preshape, the robot
ran a grasp controller to close the hand. The grasp controller
closed the fingers at a constant velocity, stopping each finger
independently when contact was detected by the measured
joint velocities being close to zero. The grasp controller closed
60

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2020

the second and third joints of the nonthumb fingers and the
two distal joints of the thumb. Note that the proximal joint of
all nonthumb fingers rotated the finger about its major axis,
causing it to change the direction of closing. As such, we maintained the angle provided by the grasp planner for these joints.
Upon closing, the robot attempted to lift the object to a
height of 15 cm. If the robot succeeds in reaching this height
without the object falling, the simulator automatically labels
the grasp as successful. We collected 10,809 grasp attempts in
total, of which 2,804 resulted in successful grasps. The data set
covers more than 100 objects of the Bigbird [34] data set,
which contains more than seven times the number of grasp
attempts as in the data set from our previous work [24],
including 1,898 successful side grasps and 906 successful overhead grasps. We used 8,988 grasps for training of our grasp
model and 1,811 grasps for testing. Table 1 shows the number
of successful and failed grasp attempts of side and overhead
grasps separately in our training and offline testing sets.
Grasp Model Training
We trained our networks using the Adam optimizer with
minibatches of size 64 for 90 epochs. The learning rate started
at 0.001 and decreased by 10 # every 30 epochs. The MDN
was trained with the same specifications. The training of both
models requires fewer than 25 min on a computer with an
Intel-i74790k processor, 64-GB RAM, and an Nvidia GeForce
GTX 970 graphics card. We implemented all our deep network models in TensorFlow. We fit the GMM prior parameters using the expectation-maximization algorithm over all
grasp configurations attempted in the training set.
We pretrained the voxel encoder for our classifier and
MDN on a voxel-based 3D object-reconstruction task. We
froze the voxel encoder parameters to the values learned on

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020