IEEE Robotics & Automation Magazine - June 2020 - 74

hyperparameter values become more consistent near the
optimum, rather than being scattered.
Overall, the optimization favored network architectures
(Table 3) that had a deep convolutional stage (five hidden
layers); a shallow, fully connected stage (one hidden layer);
the most convolutional filters allowed (512 per layer); a
modest number of dense units (16); and ReLU activation
functions. For the training hyperparameters, small batch
sizes (16) were preferred, likely because these data are regular enough to not require many samples to find a reasonable
stochastic gradient. Also, larger batch sizes can sometimes
exceed the GPU's memory, resulting in a failed run. Dropout was hardly used (0.001), which may be because the convolutional layers were already heavily regularized due to
weight sharing and the number of dense layers and their
sizes were already constrained. There was little L1 regularization but some L2 regularization (0.001 and 0.064), which
again might be because it was better to constrain the number and sizes of the hidden layers.
Tactile Pose Estimation for a 3D Edge
3D Edge Model Results
The optimized PoseNet performance was assessed for the 3D
edge on a third data set, consisting of 2,000 tactile images
labeled with the horizontal pose, depth, roll, pitch, and yaw.
The data were again subject to unlabeled pose perturbations,
with the poses and perturbations sampled randomly (Table 2).
Accurate performance was obtained (Figure 9), with MAEs
between the predicted and labeled pose parameters of 0.6 and
0.2 mm and 1.4, 1.6, and 6.4º. Clearly, these are less accurate
than the surface, which reflects the fact that pose prediction
for the edge is a far more difficult regression problem. For
example, the edge has two more pose parameters than the surface, but the same amount of training data was used. Also, by
the nature of its geometry, the edge can feel similar across

Table 3. The optimal PoseNet hyperparameters.

Parameter

3D Surface

3D Edge

Number of convolutional hidden
layers, N conv

Number of convolutional kernels,
N filters

512

256

Number of dense hidden layers,
N dense

Number of dense hidden layer
units, N unit

512

Hidden-layer activation function

ReLU

Dropout coefficient

0.001

0.203

L1-regularization coefficient

0.001

0.0001

L2-regularization coefficient

0.064

0.0003

Batch size

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2020

distinct poses: One example is that contacts at the horizontal
extremes are ambiguous under the yaw. These difficulties
emphasize the importance of finding the best possible model
fit using hyperparameter optimization.
Analysis of 3D Edge Model Optimization
For the 3D edge model, the hyperparameter optimization
resulted in a 10-fold reduction of the loss across the 300 training trials, with the lowest losses after trial [Figure 10(a) and
(b)]. This improvement in the loss was substantial but much
less than for the 3D surface, which again reflects the fact that
estimating the pose of the edge is a more difficult problem.
The losses were in the range 0.1-1 MSE, with the lowest losses
occurring only near the end of the optimization. The convergence of the optimization can also be seen in scatter plots of
the hyperparameters (Figure 10), with the lowest losses
bunched at the left of the plots.
Overall, the optimization favored network architectures
(Table 3) that had a deep convolutional stage (five hidden
layers), a fairly shallow fully connected stage (two hidden
layers), many convolutional filters (256 per layer), a moderate number of dense units (64), and ReLU activation
functions. For the training hyperparameters, dropout was
strongly preferred (0.2), and L1 and L2 regularization were
rarely used (K 0.001). Modest batch sizes were again preferred. The network and training hyperparameters were
similar to those for the surface, apart from a greater use of
dropout with the ReLU activation function. This appears
to be a strategy for coping with the more difficult pose
estimation problem for the edge. The different configuration emphasizes the benefit of an automated process for
model optimization.
Demonstration of 3D Object Exploration
Pose estimation is a fundamental capability of tactile sensing because knowledge of the relative pose enables a robot
to control its interactions. The local pose gives information about how to reposition the tactile sensor to maintain
contact while moving safely across the object. To show
pose estimation in action, Figure 1 displays trajectories
generated by the PoseNet models in this article applied to
controlling a robot moving across a complex 3D surface
and edge.
We used a robot system consisting of a TacTip mounted
on a 6-degrees-of-freedom robot arm (IRB 120, ABB robotics). This robot was previously used to study 2D contour-following [1]; we refer to that paper for details of the robot,
software infrastructure, and literature on tactile servoing. To
demonstrate 3D surface following, we extended the previous
2D control policy to a 3D surface (three pose variables) and a
3D edge (five pose variables). This policy has two aims: 1)
move the sensor to remain normal to the object surface and
2) move the sensor tangentially along the surface (by 1 mm
per time step t). Here, we implemented a proportional-integral controller in discrete time with, as output, a change in the
3D pose of the sensor in its reference frame:

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020