IEEE Robotics & Automation Magazine - September 2013 - 55

Experimental Setup
The experimental setup includes a PMD CamCube ToF camera and a PointGrey Flea camera rigidly attached to the last
link of a Barrett WAM arm (Figure 1). The PointGrey Flea
camera is, however, not used in the experiments here. As can
be observed, the cameras are displaced from the robot endeffector position to leave room for a cutting tool that we have
designed for the given task.
We have opted for a configuration where the cutting tool is
outside the field of view of the camera. This implies that, during the robot motion from the close view of the leaf to the
placement of the cutting tool, the leaf is not in the camera's
field of view, and the motion is then executed in open loop.
Implicitly we assume that the leaf will not move and that the
robot has enough precision along this small motion.
The robot and plant initial relative configuration assures
that the plant's region of interest is reachable by the robot's
cutting tool. In a similar way, plant position is guaranteed to
be inside the field of view of the camera's initial pose. In the
close view, the camera is placed in a frontal configuration at
40-cm distance from the localized leaf.
Basic Verification of the Method
The presented robotic leaf-probing strategy assumes that,
for successful sampling of plant leaves, it is advantageous to
move first to a closer and fronto-parallel viewing position
with respect to the leaf surface. To support this claim, we
verify that 1) surface normals of leaves can indeed be accurately estimated with the given method, and 2) moving to a
closer, fronto-parallel view of a leaf allows better verification of suitable leaves for probing and thus a better determination of grasping points.
To test assumption 1), we used a planar artificial leaf. For
this purpose, the shape of a real leaf (Anthurium) was taken
and cut from a carton. The shape was also used as a model
leaf for this particular experiment. The artificial leaf was
attached to a beam and rotated around its center to attain different angles of its surface normal with the viewing direction
of the PMD camera. The leaf was rotated in steps of 5°, starting at 0°, and a depth and an infrared-intensity image was
acquired at each step. Using our method, the leaf was segmented and the surface normal was computed by fitting a
plane to the 3-D points of the segment. The enclosing angle of
the measured surface normal with the z-axis was calculated.
Fitting of a line to the data revealed an approximate measurement error of about ! 0.7c, which demonstrates that a sufficiently accurate estimation of the surface normal can be
obtained with the system.

0.9
0.8
0.7
Validity

leaf. The approach vector is normalized and used together
with the grasping point to compute the intermediate goal position x goal = x g + 10a g , at 10-cm distance from the edge point
toward the outside of the leaf.
We further define a probing point at which the tool should
be finally placed x probing = x g - 2a g . The probing point is
located 2 cm from the edge point toward the inside of the leaf.

0.6
0.5
0.4
0.3
0.2
0.1

30
40
Angle (°)

Figure 5. The validity of the leaf as a function of the measured
enclosing angle of the surface normal with the z-axis
(camera-viewing direction) for a real leaf (Anthurium). The
validity measures the correlation between the measured and
transformed 2-D segment contour and a 2-D model-leaf contour.

To verify assumption 2), we used the model-leaf contour
to calculate the validity of the extracted segments during the
previous experiment. The validity measures the correlation
between the measured 2-D segment contour and a 2-D
model-leaf contour (see the section describing the extraction
of grasping points). We observed that the validity decreases as
the angle increases, i.e., the further we move away from the
fronto-parallel position, the harder leaf recognition becomes
because of view-dependent shape distortions and other visibility impairments. This also implies that the grasping point
cannot be accurately determined past some angle because the
model-leaf contour together with the associated grasping
point will fit the segment boundary only very poorly. We further acquired depth and infrared-intensity images from various viewing angles of a real leaf (the very leaf that had also
been used to extract the model-leaf shape in the previous
experiment). The computed validities, displayed in Figure 5,
are similar to the validities obtained for the artificial leaf. The
experiments demonstrate that the method is capable of
extracting the target leaf despite varying viewing conditions
and without adjusting the parameters. A close view for probing is desirable since it increases the amount of data that can
be gathered about a leaf (the resolution), which is immediately evident and does not need to be demonstrated.
Combining the data for the artificial and the real leaf
showed that the validity follows roughly a linear relationship.
By finding the mean distance of the measured points to the
fitted line, we obtained an approximate error measure of the
computed validity values of about ±0.1. However, this measure has been computed for leaves that are mostly planar.
Bent or curled leaves might show larger errors.
Leaf Probing
After verifying the basic assumptions of our approach, we
test the method on different plants and for different viewpoints. Each experiment proceeds in the same way. First, the
september 2013

IEEE ROBOTICS & AUTOMATION MAGAZINE

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2013