IEEE Robotics & Automation Magazine - March 2012 - 78

initiative facility, and we are working to establish a shared
and growing repository of consistently annotated/analyzed
imagery that is readily accessible to end users and suitable
for training of machine learning algorithms.
Georeferenced Imagery
One of the primary data streams being collected by the
vehicle are the high-resolution images of the seafloor. It is
important that these images are tagged with sufficient
information to allow our end users to georeference their
location and conduct principled scientific analysis of their
content. Georeferencing of the images allows the observations to be related to other information being collected by
the vehicle and ship-borne systems. Each image is delivered as a geotiff that is tagged with its geographic coordinates. This allows the images to be loaded into standard
geographic information systems to be integrated with
other data types. We also provide additional information
relevant to the interpretation of the imagery including
depth, altitude, vehicle pose, and oceanographic variables
associated with each image.
Seafloor 3-D Reconstruction and Visualization
Although SLAM recovers consistent estimates of the vehicle trajectory, the estimated vehicle poses themselves do
not provide a representation of the environment suitable
for human interpretation. A typical dive will yield several
thousand georeferenced overlapping stereo pairs. While
useful in themselves, single images make it difficult to
appreciate spatial features and patterns at larger scales. We
have developed a suite of tools to combine the SLAM
trajectory estimates with the stereo image pairs to generate
3-D meshes and place them in a common reference frame
[12]. These meshes are generated once the vehicle is recovered and take the same amount of time to compute as the
length of the dive allowing dive outcomes to be examined
while still at a site. The resulting composite mesh allows a
user to quickly and easily interact with the data while
choosing the scale and viewpoint suitable for the investigation. In contrast to more conventional photomosaicking
approaches [18], [19], the full 3-D spatial relationships
within the data are preserved and users can move from a
high-level view of the environment down to very detailed
investigation of individual images and features of interest
within them. This is a useful data exploration tool for the
end user to develop an intuition of the scales and distributions of spatial patterns within the seafloor habitats.
Examples of the detail achieved in the meshes derived
from the data collected as part of urchin barrens surveys in
Tasmania are shown in Figure 3 [10]. The top subfigure
shows a segment of the dense reconstruction texture
mapped using the color imagery. The striping evident in
the texture maps is a result of differences in illumination
during reciprocal legs of the survey. Also shown are the
stereo-derived bathymetric surface model onto which the
texture map is projected and the detailed views of a
78

IEEE ROBOTICS & AUTOMATION MAGAZINE

MARCH 2012

segment of the mesh. Both the boulders in the field and
patches of kelp are evident in the resulting surface.
Sample reconstructions produced using data collected
during recently completed surveys in Western Australia
are shown in Figure 4. While it is possible to examine the
individual images that were used to generate these 3-D surface models, the spatial structure of each habitat is more
evident in the composite mesh. It is also more straightforward to identify common elements of these meshes when
examining the surveys across years as gross features can be
used to guide the visual inspection of the meshes. By providing the ability to not only collect images over the same
area of the seafloor but to quickly identify common features, it is possible to identify changes within the survey
site. Figure 5 shows two examples of sites that were revisited across a year and illustrate the changes we were able to
detect using this approach to benthic observation.
Image-Based Habitat Classification
While the visualization of detailed 3-D reconstructions
improves our ability to understand the spatial layout of
seafloor features, further analysis and interpretation of the
data gathered during a dive is required to address tasks
such as habitat characterization and monitoring. This
analysis stage is typically performed by human experts
which limits the amount and speed of data processing [20].
It is unlikely that machines will match humans at fine-scale
classification any time soon, but machines can now perform preliminary, coarse classification to provide timely
and relevant feedback to assist human interpretation and
focus attention on features of interest. We are developing
image-based habitat classification and clustering systems
to facilitate the analysis of the large volumes of image data
collected by the AUV [21], [22].
We are also investigating techniques suitable for classifying habitats when little or no a priori training information is available [22]. We have developed methods based
on the variational Dirichlet process (VDP) that allows very
large volumes of image data to be clustered in a fully
automated manner. We have explored suitable features
for such clustering, including color, texture, multiscale
measures of rugosity, slope, and aspect (or orientation)
derived from fine-scale bathymetric reconstructions created using georeferenced stereo imagery collected by an
AUV [23]. An example of the application of these techniques to data collected in South East Queensland is shown
in Figure 6. The images from the broad-scale, sparse grid
surveys were clustered based on color, texture, and rugosity queues extracted from the stereo imagery. The VDP
parameters learned from the broad-scale dives were then
used to classify the observations from all of the dense dives
from the northern and southern regions of the survey site.
These results have been plotted over the vehicle track and
have been combined with a geotiff of the depth contour in
the area in Figure 6. It is interesting to note that the habitat
distributions are strongly correlated with depth despite the

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2012