IEEE Spectrum September, 2013 - 41

Photos, CloCkwise froM right: abner kingMan/aCea; riCardo Pinto/aCea (2)

Achieving 2-cm accuracy requires one more thing.
We've added a base station onshore; we surveyed the
spot and identified its precise location. Because the reference GPS receiver at our base station knows exactly
where it is, it can estimate the effects of atmospheric
conditions that cause signal delays. It can then calculate corrections for those delays and send them to
the GPS receivers on the boats; those corrections are
necessary to get that 2-cm position accuracy.
There are currently 31 satellites operating as part
of the GPS system; a receiver typically needs to be
able to receive transmissions from four of them in
order to pinpoint its location. On occasion, a boat's
own carbon wing or the wing of a nearby boat blocks
so many satellites that we lose the ability to track its
position via GPS. That, for us, would be a disaster.
So we needed yet another backup plan. We selected
an off-the-shelf navigation system that couples GPS
with the inertial navigation system. An INS uses inertial sensors to implement the most basic navigational
scheme there is: dead reckoning. The system has internal accelerometers and gyroscopes that constantly
measure linear and rotational movement and direction; it also has an onboard processor to calculate
navigation information using that data. With an INS
on board, the crew no longer need compasses, but
they carry them anyway.

BAcK OnSHOre, a computer system takes all the position data from the
boats and the helicopter-also outfitted with our augmented GPS-INS system-
along with the video feed from the camera and matches every location in the
real world to a pixel in the camera image. This enables us to place the laylines
and the tags identifying the boats and their speeds precisely into the image so
that it looks realistic to the viewer.
An enormous amount of calculation is necessary to make that happen. But
it's based on some of the most basic principles of fine art. If you've ever taken
a basic art class, you've studied perspective. First, you pick a "vanishing point"
at what will be a distant place in your drawing. To position an object in the
scene, you draw straight lines from the front of the object to the vanishing
point, and then align the rest of the object along those lines.
Here, we've turned that around. Our single distant point is the camera. It corresponds, if you will, to the vanishing point-in this case called the point of view.
Our system takes all the pixels in the image and calculates imaginary lines that
connect each pixel to the camera. Now, say we want to put a boat's name near
the boat. We rely on the position data to identify the correct object-the boat-
and then use those calculated perspective lines to position the name in relation to the boat so it appears integrated into the scene in a visually pleasing way.
In addition to the data needed for the augmented-reality system, we're streaming 80 megabits per second of video from up to four of the seven HDTV cameras
mounted on each boat, and we're sending signals from shore that allow cameramen to remotely adjust each camera's pan, tilt, zoom, and focus. We also send
about 500 kilobits per second of data to and from the boats, including things
like race boat locations, course boundary changes, locations of course marks,
and penalties, and about 150 kb/s to and from the helicopter. We'll be sending
data in a section of the 2-GHz band that we licensed for this purpose, using a
modulation technique called coded orthogonal frequency-division multiplexing.
COFDM spreads a digital transmission across multiple subfrequencies within
the allocated frequency band; it's the way digital broadcast television works in
most of Europe. It is also the modulation system used in one of the most advanced Wi-Fi standards, IEEE 802.11n. But we're using a time-slotted protocol,
which is different from conventional Wi-Fi.
Typically, today's Wi-Fi and Ethernet systems detect in real time when a data
packet runs into interference from another data packet sent from another user;
the first transmitter then waits a random amount of time and resends the packet.
This method is often called carrier sense multiple access with collision detection,
or CSMA/CD. In time-slotted protocols, transmissions aren't random but rather
scheduled to take turns using the channel. Time-slotted protocols are an older
approach that isn't used widely today because they're less flexible and, in cases
of unpredictable channel usage, less efficient. But in our case, with steady data
rates, time-slotted techniques avoid collisions and we end up gaining in efficiency.
We send the video separately on licensed frequencies in the 3-GHz band. There
are up to 14 HD video signals coming from the racing yachts, chase boats, various
official boats, and helicopters, and each of those HD signals will be compressed
from 1500 down to a range from 5 to 10 Mb/s and transmitted using COFDM.
The magic really happens when we combine the position information from
sensors on board the boats with the video and camera position data from the
helicopter. Viewers will be able to see not only tiny boats inching along the
TV screen but also lines, shapes, and other graphics that appear to be painted
on the rolling water.
Creating this illusion starts with putting a time stamp on each frame of the
video as it is shot; we'll also be time-stamping the position data from the race
boats. We'll use a high-speed PC with a specialized video capture card to bring
graphics based on the position data and live video together in a composite
spectrUm.ieee.orG

north american

sep 2013

http://spectrUm.ieee.orG

Table of Contents for the Digital Edition of IEEE Spectrum September, 2013