ADAS & Autonomous Vehicle Engineering - November 2023 - 26

Sensors
The rFpro system can accurately render challenging images, such as varying
light conditions at night.
While humans sample what we see in one go, a
camera sensor samples it line by line down a chip, like
a cathode ray tube television screen that builds the
picture line-by-line. It means that each pixel is sampled
at a slightly different time. That in turn means that it
can't be sampled in real time. " You get a 'rolling shutter'
effect " , says Daley, " So, when you have fast-moving
objects, it creates a motion blur. It is critical to replicate
this phenomenon as that is how the sensor 'sees'
the world. That's why we took a blank sheet of paper
and started again for a dedicated ray traced engine
designed for sensor simulation. "
It is not purely about ray tracing. Light from the
vehicle needs to be simulated as well as light from the
sun and other light sources on the road. This has proved
useful to headlight manufacturers who have used the
simulation technology to reduce the required amount
of road testing.. It has led to a reduction in working
with prototype headlights on actual roads and reduced
the time engineers work during the hours of darkness.
Applicable to lidar
Models can be built for lidar too. " Lidar on the vehicle is
moving around, it's shooting individual rays, individual
laser beams into the scene to collect that distance information
and reflectivity about what it is hitting. In the
same way that light has a level, lasers come back with a
reflection strength, so you can build all that into a simulation
and really replicate the physical devices, " says Daley.
Because digital images are built up from multiple
sampled images, this introduces other issues when
dealing with light sources that are not constant, such as
LEDs. Those used in traffic lights flicker at about 200Hz,
for instance, so if you are sampling the scene with exposures
of one or two milliseconds, the traffic lights flickering
every five milliseconds won't always be lit in the
images generated. If the sampling rate is lengthened, to
say 10 milliseconds per exposure, it will compensate for
the flicker, but will then introduce motion blur, either
from a static source with passing objects or a moving
source when sampling static objects, which, in simulated
testing would need to be simulated too.
" You can't just post-process an image and give
something motion blur " , said Daley. " It doesn't appear
in the right way. You have to model it into the way that
you're sampling the world. You're keeping the pixels open
for a period of time, and things move. So, you often see it
at the edge of images where the relative movement next
to you is fast, so your own motion, or where vehicles
are passing in front of you. Motion blur depends on
the type of imager, so some imagers have what we call
a global shutter. They sample the world for all lines at
the same time. Most of them don't, most of them have
rolling shutters where the top line of pixels is sampled
a microsecond before the next line and a microsecond
before the next line, so by the time you've got from the
top to the bottom of the image you're at quite a significant
amount of time difference, so a pillar, a pole actually
becomes slanted in the image because you've passed it.
" Now, if you're training a safety-critical system
to recognize the world, you can't ignore some of the
fundamental problems you have with the way that they
are engineered, so you have to model that in. Again,
that is where the ray tracer helps. It's about enabling
us to sample the world in exactly the same way as a
physical chip does, " Daley explained
" The ray tracer has given us the sensor's image as
accurately as possible. We need to give customers the
answers as well, so you have to create bounding boxes, very
simplistic sets of training data, 2D bounding boxes. Then
you also do things like semantic segmentations. People
are manually labelling images, they're taking half an hour
per frame to draw around a set of pixels to say, 'That was
a car.' In simulation you get it instantaneously, so we're
able to not only produce the highest quality sensor data,
but also the most accurate and the cheapest way to label
it. The combination of the two is immensely powerful. " ■
26 November 2023
ADAS & AUTONOMOUS VEHICLE ENGINEERING
26 November 2023
ADAS & AUTONOMOUS VEHICLE ENGINEERING
rFpro
ADAS & Autonomous Vehicle Engineering - November 2023

Table of Contents for the Digital Edition of ADAS & Autonomous Vehicle Engineering - November 2023
