IEEE Robotics & Automation Magazine - June 2020 - 61

this reconstruction task during grasp training of the classifier
and MDN. We found that freezing the encoder parameters
achieves better testing performance than fine-tuning the
encoder parameters for both models.
Our voxel reconstruction autoencoder has the same output and decoder structure as the reconstruction variational
autoencoder in Brock et al. [35], except we dropped the variational constraint, i.e., we do not predict a variance associated
with the output of the encoder. We found this achieves better
reconstruction results than the variational autoencoder in
Brock et al. [35] on our data set.
To train our voxel reconstruction autoencoder, we synthetically rendered 590 meshes from the Grasp Database [36] at
200 random orientations each, adding noise to the depth
images to reflect sensor noise. We backprojected these points
into a 3D pointcloud, which we voxelized to a 26 # 26 # 26
voxel grid, centered in a 32 # 32 # 32 total voxel grid. We
scaled the bounding box extracted from the segmented object
pointcloud to have 26 # 26 # 26 voxels. We were here concerned only with capturing object-shape information, as
object size and pose were handled by the grasp network independently. As such, we learned the mapping from the partial
pointcloud voxelization to a centered and independently
scaled full-mesh voxelization. We trained with a sigmoid
cross-entropy loss on the true and predicted reconstructed
voxel representation. The voxel reconstruction autoencoder
was trained using the momentum optimizer with minibatches of size 64 for 100 epochs. We used a starting learning rate
of 0.001 and decreased the rate by 10 # after 20 and 80
epochs and set momentum to 0.9. Our network trained in
about 4 h on a single Nvidia Tesla V100 graphics card. Our
pretrained, voxel reconstruction network achieved 95.56%
accuracy and an F1-score of 0.5166 on a test set comprising
left out models from the Grasp Database [36] and all models
from the Yale-Carnegie Mellon University-Berkley (YCB)
data set [37].
Offline Grasp-Learning Validation
We validated the performance of our voxel-based grasp-success classifier on an offline prediction task using the held-out
test set collected in simulation. We show the prediction accuracy and F1 score in Table 2. We compared our approach
(voxel-config-net) with our previous RGB-D-based classifier
from Lu et al. [24] (rgbd-config-net). We retrained the
RGB-D network on the training data of this article with the
same specifications as stated, except we used minibatches of
size 8. The RGB-D network takes more than 24 h to finish 90
epochs of training.
Our voxel-based network significantly outperformed the
classifier using the RGB-D object representation on this task
in terms of F1 and accuracy. This result holds for both side
and overhead grasps. We believe this implies the voxel-based
approach encodes the object geometry better for grasping
than using an RGB-D image directly.
We treated predictions with grasp probability above 0.5 as
positive for the voxel-based approach. As done in Lu et al. [24],

we thresholded the RGB-D network predictions with a value of
0.4, as setting the threshold to 0.5 predicts all grasps as failures.
This necessary modification highlights another advantage of
our novel voxel-based classifier: the predicted grasp probabilities better reflect the true success probabilities, making them
more useful for planning as inference.
For completeness, we also show the MDN negative log
likelihood loss on the testing set, where smaller MDN loss
reflects higher conditional probability density.
Robotic Grasp Inference Experiments
We evaluated grasp planning as inference on the physical
robot system using our learned voxel-based grasp classifier
with all three grasp-configuration prior probability models
described in the section "Grasp Planning as Probabilistic
Inference" (i.e., uniform, GMM, and object-conditional). We
performed experiments on eight YCB [37] objects covering
different textures, shapes, and sizes. We show the experimental setup and objects used in Figure 3. All experimental
objects are unseen in training except for the Pringles object.
We attempted both overhead and side grasps at five different
poses per object, for a total of 80 grasp attempts per method.
We used the same set of locations across different methods,
but each object had its own set of random poses. In total, we
Table 2. Grasp model evaluation on testing set
for all, side, and overhead grasps.
voxel-config-net

RGB-D-config-net

Grasp
Type

Accuracy

MDN
Loss

Both

0.786

0.592

0.573

0.476

−10.73

Side

0.724

0.649

0.661

0.401

−14.71

Overhead

0.842

0.456

0.474

0.52

−7.2

We show the accuracy and F1 score of voxel-config-net and RGB-Dconfig-net. We also show the negative log likelihood loss of MDN.

Figure 3. The experimental setup with the objects used. Objects
are (from left) a Pringles container, Lego construction, soccer
ball, mug, mustard bottle, sugar box, soft scrub bottle, and
pitcher. Objects range in size from 8 # 9 # 11 cm (mug) to
13 # 17 # 24 cm (pitcher).

JUNE 2020

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020