Automotive Engineering - December 2024 - 19

COVER STORY
Luminar's Iris lidar sensor features
a custom InGaAs detector, a fiber
laser, a 2-axis scanning mirror,
and a mixed signal ASIC.
matrix of data transfer and processing requirements.
And each parameter must be tuned for various cameras
around the car. For example, adjusting camera height
changes tuning choices. The sum of all these choices
must then be tuned to work with an AI perception
model-for instance, how many megapixels and how
many frames per second in a forward-facing camera are
ideal for a given model to accurately identify a person
in an image, at what speed, and at what distance?
Or take the considerations necessary for lidar, such
as the Luminar unit Volvo uses on the new EX90. Lidar
output is called a point cloud, a Seurat-like picture of
an environment made up of points. Matt Weed,
Luminar's senior director of product strategy, told SAE
Media that a sensor fusion design needs to decide on
the kind of lidar output. A three-dimensional point
cloud requires more compute compared to a two-dimensional
array that represents three dimensions.
With the latter, designers need to decide how to represent
2D as 3D. " The common one right now is a topdown
projection, like a bird-eye view, " Weed said. " You
have things called pillars, where you have attribute
height and gap and under-drivability, but not in a
highly resolved way. " The upside to this is " it's not
blowing up your [processing] computer like crazy, like
a full three-dimensional model, but it's still going to be
better than what you get in a camera. " These data-inAI-identification
calibrations must be done for every
type of sensor on a vehicle.
AUTOMOTIVE ENGINEERING
Fusion: late or early?
At the other end of fractal complexity lies the question of where sensor
fusion occurs in ADAS operations. The two broad approaches are
late fusion, also known as object-level fusion, and early fusion, also
known as raw data fusion or low-level fusion. The flow chart described
above is considered late fusion, the process automakers initially decided
on. This technique puts perception work on the sensor itself, filtering
raw data at the sensor to identify objects in the environment,
then sending those perceptions to the compute for fusion. There, the
AI model considers the fused perceptions for its world-building, validation,
and vehicle reaction planning. " This approach is simpler because
you push much of the hard work to third-party components, " Rivian
said. " But [it's] limited because the outcome is mostly as good as your
least-capable sensor... [and] objects must be detected by more than
one sensor to be considered valid. " If the camera's perception data
indicates an object ahead that the radar hasn't registered, the final AI
model doesn't have raw sensor data that it could use to make its own
decision about the variation, with potentially deadly consequences.
Today, more companies are interested in early fusion, including Rivian,
Luminar and ADAS software developer LeddarTech. LeddarTech chief
technology officer Pierre Olivier told SAE Media that early fusion allows
ADAS to use lower-cost sensors without high-performance compute
power. Because " every step of processing data is a filter, " Olivier said,
utilizing the full spectrum of raw data maximizes output quality. Rivian
said this provides, " richer and more accurate representations. " Luminar's
Weed said this method is standard for lidar units, because every lidar
maker is " trying to keep power budget down in the sensor, which has to
live at the extremity of the vehicle and deal with heat. "
December 2024 19
LUMINAR
Automotive Engineering - December 2024

Table of Contents for the Digital Edition of Automotive Engineering - December 2024
