IEEE Robotics & Automation Magazine - September 2023 - 75

distinct stages for adding a point to the
cloud: 1) input preprocessing, including
cropping of the input RGB-D points to a
sufficiently small volume to include only
the turntable and object to be scanned; 2)
registration using an iterative closest point
algorithm; and 3) integration, i.e., the
decision-making module responsible for
accepting or rejecting points into the
point cloud. We rotated the objects during
the registration process using a turntable
to achieve 360° surface coverage.
The collection of the visual data,
V ,
"
BY EXPOSING THE NETWORK
TO RANDOM
ORIENTATIONS, WE
ENCOURAGE THE NETWORK
TO LEARN POSEAGNOSTIC
EMBEDDINGS.
„
was
repeated twice to gather two sample
point clouds for each object: one for training
and another for validation. Subsequently, we manually
removed the background from each visual point cloud and
aligned its z-axis with the normal alignment of the tabletop.
The total number of points in the vision point clouds ranged
from 619 for the Rubik's cube, to 10,980 for the camera box.
CURRICULUM SAMPLES AND SCHEME
For the cur r iculum scheme we create a dataset of
pa r tial point clouds
V ,2000 as described in the " CL "
section,
to create 2,000 samples for each proportion
p {. ,. ,, .}. For the partitioning, we create a
cluster size mV 10 .= The value was elected empirically: a
! 0050110g
small value leads to clusters containing a small amount of
information, and a high value reduces the number of possible
partial point cloud we can generate. We decided to select it
such that the vision clouds formed at least 40 clusters, considering
that the smallest filtered vision point cloud is the golf
ball, with 408 points. As the voxel filter equalizes the spatial
density, the clusters roughly represent equal areas across
models. The curriculum reduced the proportion p in later
epochs; specifically, the network was trained with
p {. ,. ,. ,. }
! 10 07 04 02
71-75], respectively.
for epochs [1-40, 40-55, 56-70,
PARTIAL TACTILE POINT CLOUDS
Similar to the vision, the dataset T was decomposed to create
a dataset of partial point clouds,
tial
the network can be viewed as few-shot
learning, or loosely as one-shot learning
[20]. As is common in few-shot learning,
the training datasets were augmented with
additional samples derived from the original
point clouds to reduce the risk of overfitting
to the visual training set.
Data augmentation was performed
according to the transforms in Table 1. For
this specific implementation, the input
space of PointNet was fixed to b = 1,024
as during training, samples need to be
of equal length when grouped into batches.
Then, the first step, which was also
the first introduction of variation, was the
uniform selection (without replacement) of b points from the
clouds of shape (, ).3a
For point clouds with fewer than 1,024
points, duplicate points were added to pad up to the correct
shape. Subsequently, zero-mean Gaussian noise with a standard
deviation of 0.5 mm, N( ,. ),00 52
was added to each point
and the resulting point cloud rotated by a random 3D Euclidean
transform. By exposing the network to random orientations,
we encourage the network to learn pose-agnostic embeddings.
Finally, a random scaling was applied from the uniform distribution
(. ,. ).
U 0951 05 Overall, this training scheme is designed
to introduce variance through random scaling, additive noise,
and random rotations to reduce the likelihood of overfitting or
memorizing the arrangements of the points.
For clarity, the transforms used during validation and evaluation
are shown in Table 2. Additive noise and scaling were never applied
to the visual validation or tactile evaluation stages.
TRAINING AND EVALUATION
We follow the training of PointNet as in [10], with crossentropy
loss and an Adam optimizer with a learning rate of
TABLE 1. Visual data augmentation during training.
STEP TRANSFORMATION
Input point cloud
T ,2000 containing partactile
representations of the objects at different
proportions p. A value of mT 7= points was chosen for a
cluster size similar to the end-effector taxel number. In this
way, we considered each patch Pk to be generated by a single
touch. Therefore, the resulting partial point clouds are
generated randomly by composing small sets of clusters,
each representing a small surface comparable to the size of
the CySkin sensor.
ONE-SHOT LEARNING-DATA AUGMENTATION
In addition to the training curriculum, we applied data augmentation
techniques at runtime to both V and
V .2000 As the
number of visual models was only one training point cloud and
one validation point cloud per object, the challenge of training
1
2
3
4
Random selection of b points
Additive white Gaussian noise
Random 3D rotation
Random scaling by scale factor in the
interval [0.95,1.05]
b # 3
b #3
b # 3
b # 3
DIMENSIONS
a # 3
TABLE 2. Visual data preprocessing during validation.
STEP TRANSFORMATION
Input point cloud
1
2
Random selection of b points
Random 3D rotation
b # 3
b # 3
DIMENSIONS
a # 3
SEPTEMBER 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
75
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
