IEEE Robotics & Automation Magazine - September 2023 - 74

(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
FIGURE 6. The 10 objects in our datasets. (a) A baseball, (b) beer, (c) a camera box, (d) Rubik's cube, (e) golf ball, (f) an orange;
(g) a pack of cards, (h) shampoo, (i) Spam, and (j) tape.
The objects can be seen in Figure 6. Each object resides in
three approximate shape classes: cuboidal, cylindrical, and
ellipsoidal. The selection contained some geometrically similar
shapes, such as a golf ball and baseball, and different
materials, such as metallic, plastic, or cardboard, which also
have vastly different surface textures and reflective properties.
The selection of objects was limited to rigid, nondeformable
items.
CYSKIN AND COLLECTION OF TACTILE DATA
The robot setup consisted of a seven degree-of-freedom
Panda by Franka Emika industrial manipulator, with a
CySkin module attached to the gripper. CySkin is a capacitive-based
tactile-sensing technology. The sensor patch integrated
into the end effector used in the experiment included
seven tactile elements arranged as shown in Figure 7(b). Each
tactile element (taxel) has a 3.5-mm diameter and provides a
16-bit measurement, which is related to the contact pressure.
The pitch among adjacent taxels is 7.5 mm. A single contact
between the end effector and the object activates one or more
tactile elements when the response of the taxels exceeds a
threshold value. The tactile point cloud is subsequently created
by registering the 3D position of the active tactile elements
with respect to the robot's reference frame.
The collection procedure began with fixing each of the 10
objects, in turn at a known position in front of the robot [see
Figure 7(a)]. Next, an operator manually guided the robot to
touch the object with the sensorized end effector. The robot
was manually moved in small steps, incrementally exploring
the surfaces until reaching full coverage of the object. As the
goal of the procedure was to cover the whole surface of the
objects thoroughly, the operator mainly applied small movements
to the end effector. This approach biased the touches
toward the local areas, in contrast to exploration strategies that
make larger movements or seek to maximize the information
gain per touch.
Some surfaces of the objects were not reachable due to the
limits of the setup. These limitations are the following: 1) the
planar tactile-sensing technology was not able to capture tight
manifolds with an extreme concave curvature, such as the rim
of the beer can; 2) it was not possible to explore the inside
surface of the reel of tape as the dimensions of the end effector
were larger than the confined space; 3) the lower surfaces
of the spherical objects (baseball, golf ball, and orange) and
the aspects covered by the jig fixing the objects in the environment
were obstructed from exploration, resulting in hemispherical
representations.
The explorations were repeated three times and resulted
in 30 tactile point clouds, which compose the dataset T . The
point clouds ranged from 624 points for the golf ball up to
3,946 for the camera box (the dataset is provided as supplementary
material; please see the " Acknowledgments " section).
In this work, the problem of performing an autonomous exploPanda
RealSense
Camera
Manipulator
7.5
mm
CySkin
3.5 mm
Fixed
Object
(a)
(b)
FIGURE 7. (a) The robot setup as seen with the Intel RealSense
and the Panda Manipulator poised in front of the baseball,
secured by a jig. (b) The CySkin patch of seven taxels fixed to the
end effector. Annotated is both the pitch (the distance between
taxel centroids) and the diameter of each taxel.
74 IEEE ROBOTICS & AUTOMATION MAGAZINE SEPTEMBER 2023
COLLECTION OF VISION DATA
An Intel RealSense D415 RGB-D sensor was used to capture
successive images of each object. The algorithm used to
stitch together frames was adapted from the Point Cloud
Library in-hand scanner application. The algorithm has three
ration of the objects is not considered. This would add additional
challenges that are outside the scope of this work, which focuses
on partial recognition. Indeed, an exploration procedure based on
tactile sensing depends on the end-effector type, object type, and
its relative position to the robot. Furthermore, nonplanar objects
increase the complexity as some parts can be difficult or impossible
to reach. Moreover, beyond the additional challenges related
to control aspects, a proper exploration strategy is key to minimizing
the number of contact interactions required to recognize
the object with high confidence. These two aspects are currently
beyond the consideration and scope of this work. Instead, this
article focuses on analyzing the data to evaluate the possibility
of recognizing the objects from partial representations. What is
developed in this article will be used in a future extension of the
method to design a proper exploration strategy driven by the confidence
obtained when processing partial tactile point clouds.
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
