IEEE Robotics & Automation Magazine - September 2018 - 97

To determine the exact cause, the estimated pose from the
homing beacon observation would be stored and Cataglyphis
would drive to its next waypoint to perform another observation. If Cataglyphis was unable to detect the homing beacon by
the time it reached the designated homing waypoint (located
5 m in front of the homing beacon), a square update was performed with the assumption that the robot was lost. In this
case, Cataglyphis would drive a 14-m square pattern centered
on the perceived location, and, while stopped at each corner, it
would search for the homing beacon. If two consecutive estimates from homing beacon observations were consistent (i.e.,
within 2°), then Cataglyphis' pose was updated.
A homing timer was also implemented to indicate to Cataglyphis that it was time to return home for a navigation update.
This was done to limit the amount of dead-reckoning error
accumulated during periods of exploration, enabling more
accurate navigation and greatly reducing the chance of getting
lost. The homing timer did not pose a hard deadline, i.e., it did
not directly interrupt ongoing robot behaviors. Instead, the
urgency to go home gradually increased with time since the last
homing visit, which was factored into the decisions made by
Cataglyphis for the next courses of actions.
To provide mapping capabilities and to have a backup
mechanism of recovering if the primary navigation approach
fails, a variant of sparse point cloud SLAM [3] was implemented onboard Cataglyphis. In this approach, the lidar-generated
3-D point cloud was compressed into a two-dimensional
(2-D) occupancy grid, with 1-m resolution after leveling
and ground removal steps. Each grid cell contained the mean
of the planar projection (xy coordinates) of all points associated with that cell and the average height information (z coordinate). Using the grid map, iterative closest point (ICP) [4] was
performed to estimate the change in robot translation from the
previous time step, whereas the change in rotation was extracted from gyro integration.
To reject incorrect correspondences, a scoring system was
developed to determine whether ICP or wheel odometry provided a more accurate solution. The previous scan was transformed to the current pose using the transformations from
both ICP and wheel odometry. The points in both scans were
then matched with the points in the current scan using the
nearest neighbor search, and the scores were calculated for
each as the inverse of the sum of absolute position and height
error between corresponding grid cells. The higher-scoring
solution was kept as the transformation between frames.
To generate a global map, a keyframe strategy [3] was
implemented. First, temporary keyframes were generated
based on the grid map every 5 m as anchors for propagating
Cataglyphis' pose. Specifically, the transformation between
the current robot pose and the previous temporary keyframe
was solved with a graph optimization tool [5], using the estimated pose transformations between consecutive time steps
as well as the transformation between each time step and the
keyframe as edges. This temporary keyframe would become
permanent if no other permanent keyframes existed within a
20-m radius, and it would be used for generating a global

map. Due to the reliability of the homing update, Cataglyphis
did not perform loop closure; instead, the homing update
was used to reset the pose estimated by SLAM.
Overall, Cataglyphis was able to reliably localize itself anywhere on the competition field with a meter-level accuracy.
When Cataglyphis updated its pose in a close distance (<5 m)
in front of the homing beacon, it had centimeter-level positioning accuracy and less than 0.3° error in heading angle. This
performance ensured that Cataglyphis could both successfully
navigate to ROIs at the far reaches of the field and deposit collected samples on the starting platform with high accuracy.
Another responsibility of the robot perception system was
to find samples, which provided a different set of challenges.
First, Cataglyphis had to be able to recognize samples with only
partial information without prior exposure (e.g., the hard samples) in a field that contained objects that closely resembled the
samples. To deal with this issue, a wide range of mock samples
were fabricated and mixed with random items that could conceivably appear in the competition field. Photographs of these
objects made up the data set used for the training and testing of
the computer vision algorithms. Another challenging problem
was searching the large field quickly enough to successfully
complete the mission. This challenge resulted in the use of a circular fisheye lens in combination with a 50-megapixel camera,
enabling the surrounding area to be adequately captured by a
single image at each search position.
The computer vision subsystem was made up of three
main components: segmentation, classification, and localization. The segmentation algorithm separated pixel clusters
based on color into salient image regions called candidates.
Although segmentation was still the most time-consuming
portion of the computer vision software, we managed to
reduce the processing time by tenfold as compared to our
2015 approach [2]. Specifically, each pixel was defined by a
single byte per color channel. The color assignments for all
possible intensity values were calculated offline and easily
accessed from memory during runtime through a lookup
table. The lookup table consumed roughly 16 MB of memory,
and each pixel could be assigned with a single-memory access
by indexing the lookup table by intensity values. Before
extracting the segmented portions of the image for classification, the image was opened and closed using morphological
operations [2] to remove artifacts. The different stages of the
segmentation process are presented in Figure 6(a)-(c).
After segmentation, the candidate images were extracted and
passed to a sample classifier. The classification approach consisted of a convolutional neural network (CNN) based on AlexNet
[6], which has two convolutional layers, three fully connected
layers, and a softmax layer. The training data set included
more than 700,000 collected images. A set of candidate images and the corresponding sample types classified are presented in Figure 6(d) and (e). To better illustrate the segmentation
step prior to classification, the distribution of pixel intensities
for some candidates in Figure 6 are provided in Figure 7.
After sample classification, each candidate was assigned a
confidence value equal to the softmax output of the CNN. The
september 2018

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - September 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2018