IEEE Robotics & Automation Magazine - September 2017 - 101

250

Northing (m)

200

150

100

0
50
(a)

100

150
200
Easting (m)
(b)

250

300

Figure 4. (a) The obstacles present in the experiment, including a person, a tarp, an inflatable kangaroo, tires, a trailer, wooden crates,
hay bales, and a 1,000 L tank. (b) Results of the obstacle detection system showing true obstacles (black) and false positives (blue).
The path the robot traveled is indicated in red. The false obstacles occur mostly at the transition from the field to the road. Note that
the false obstacles were only detected for short periods of time until they were incorporated into the model.

are currently relatively costly for use in our inexpensive
modular robots. It is also desirable to sense the crop to
ensure the robot's wheels do not damage it, as well as to have
redundant navigation sensors for robust long-term operation. With this in mind, we developed a cost-effective localization system that combines novel vision-based crop row
tracking with low-cost RTK-GNSS from a consumer grade
receiver, wheel odometry, and an industrial grade inertial
measurement unit (IMU).
Visual crop row tracking is an effective method for navigation in agricultural fields (Reid and Searcy 1987 [32]; Tillett 1991 [33]; Billingsley and Schoenfisch 1995 [34];
Hiremath et al. 2014 [35]). Most methods in the literature rely
on an initial pixel-wise color-based segmentation step to
detect plant pixels under the assumption that the crop is green
with brown soil in-between. However, many tasks in broadacre agriculture are performed in the fallow period where
crop rows are composed of brown stubble against brown dirt
(Figure 5) and so are difficult to distinguish via pixel-wise
color segmentation. To perform crop row tracking in these
visually difficult conditions, we developed a segmentationfree visual crop row tracking method.
Our crop row tracking method detailed in [8] uses monocular vision to estimate frame-to-frame relative offsets of the
crop rows by tracking their parallel texture. This method
avoids segmentation and is able to guide a robot parallel to

crop rows in a wide variety of crop types and lighting conditions, including in visually challenging fallow fields without
green crop rows. An extension to this crop row tracking
method detailed in [9] identifies the location of individual
crop rows with a support vector machine (SVM) regressor
using color, texture, and structure features from the forwardfacing stereo camera pair. This extension allows the robot to
avoid driving on the crop. The SVM can be trained in the
field with a semisupervised learning procedure that requires
minimal human input. This ability to easily retrain the crop
row detection system for each field allows adaptation to virtually any crop row appearance, including visually difficult
fields such as the fallow fields mentioned previously.
The offset from visual crop row tracking is combined with
an RTK-GPS solution, wheel odometry, and an industrial
grade IMU using a particle filter. This filtered position solution improves pass-to-pass localization accuracy and allows
continued navigation during temporary GNSS outages. We
use the open source real-time kinematic library GNSS library
[30] to compute an RTK solution for the consumer-grade
GPS receiver.
Docking for Refill
Farmers currently manually refill their machinery. However,
a fleet of robots each with a relatively small payload will
require frequent refilling, which motivates the need for an
SEPTEMBER 2017

IEEE ROBOTICS & AUTOMATION MAGAZINE

101

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2017