IEEE Robotics & Automation Magazine - September 2023 - 76

0.01. We trained the network for each
experiment five times and present the run
that performed best on the vision validation
set. Our data pipeline, including the
training and evaluating PointNet, took
approximately 20 min using a GTX 1070
graphics card. We selected the filtering
parameters with a grid search evaluated
on visual monomodal accuracy. In the
end, we selected an MLS radius of 5 mm
and a leaf size of 3.5 mm for the voxel filter.
To be clear, the tactile dataset always
remained unseen during training, evaluation,
and trained-model selection.
"
THE INCLUSION OF
FILTERS IN THE VANILLA
SCHEME IMPROVED
THE PERFORMANCE
ON WHOLE POINT
CLOUDS BY 16%,
REACHING AN
ACCURACY OF 93%.
„
RESULTS AND DISCUSSION
This section evaluates the ability of the
pipeline to recognize degrees of partial tactile point clouds
and improve using the point filters and proposed CL procedure.
In this analysis, we also evaluate the performance of the
system in terms of inference time. Furthermore, we simulated
an online exploration of the objects performed as described
in the " Experiments " section when collecting the data. This
experiment is useful to 1) assess whether the proposed method
is suitable for online recognition and 2) show how the
exploration strategy affects performance. Finally, a comparative
experiment with descriptor-based methods is proposed to
benchmark our approach.
PARTIAL POINT CLOUD RECOGNITION
The first experiment focused on the effects of the filters and
learning scheme on partial point cloud recognition. We evaluated
three configurations: 1) a vanilla training scheme without
point filters, 2) a vanilla training scheme with point
filters, and 3) CL with point filters. The vanilla and the cur0.2
0.4
0.6
0.8
1
0.2
0.4
0.6
0.8 1
Proportion of Points p
FIGURE 8. Performance of three configurations of the pipeline
across a range of partial tactile representations, dictated by the
proportion p. The three configurations are 1) a vanilla training
scheme without filters, 2) vanilla scheme with filters, and 3)
curriculum scheme with filters.
76 IEEE ROBOTICS & AUTOMATION MAGAZINE SEPTEMBER 2023
riculum training schemes used different
datasets, V and
V ,2000 respectively. The
system performance was measured
against the tactile partial point clouds in
the T2000 dataset. It should be noted that
although three tactile samples were collected
for each object, when evaluating
the network on partial point clouds, we
applied the sampling scheme described
in the " CL " section. Therefore, when
p [. ,. ],
! 0050 9 the models are evaluated
on T ,2000 which consists of 2,000 unique
point clouds for each object and each
proportion p. On the contrary, when
p = 1, the models performed predictions
on a test set of three samples for
each object.
The results, as presented in Figure 8, show that the vanillatrained
network without filters performed the worst, and on full
representations recognized objects with a baseline accuracy of
77%. The inclusion of filters in the vanilla scheme improved
the performance on whole point clouds by 16%, reaching an
accuracy of 93%. The training curriculum further boosted
performance, reaching 100% accuracy on full point clouds
and
p ..$ 06 Across the range of proportions, the vanilla
scheme with filters outperformed the baseline by an average of
11%, and the curriculum bettered that by an additional 6%. For
.,
p 02$
the curriculum-trained network performed with an
^h
$$
accuracy of at least 80%. For all proportions .,p0051
the curriculum scheme produced the best object-recognition
performances from the tactile point clouds, especially
for ..p06
$$1
It is also worth noting that the curriculum strategy is the only
one capable of reaching 100% accuracy. Indeed, as previously
discussed, when considering nonplanar shapes, some of the
objects cannot be fully explored. Therefore, even when considering
the full tactile point cloud (i.e., p = 1), the system has to
make predictions using partial data as some surfaces are missing
in the tactile representation compared with vision. As a result,
the vanilla networks cannot reach 100% accuracy as they were
not specifically trained on partial representations, further showing
the effectiveness of the curriculum strategy applied to CMR.
Vanilla Without Filters
Vanilla
Curriculum
INFERENCE TIME
To gauge whether the system has the potential to be used
online, we measured the inference time of the network,
which we defined as the time taken for a prediction to be
computed for a single point cloud. We included in the measurement
voxel filtering, MLS filtering, preprocessing operations,
and forward pass of the NN. The experiment was
performed on point clouds in the augmented dataset, which
varied between 600 and 4,000 points. A batch size of one
was used, as is desirable in an online recognition system,
where the data are updated after each contact.
The pipeline was able to make a prediction in a mean time
of 21.4 ms for a single point cloud, corresponding to a refresh
Accuracy
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
