Tech Briefs Magazine - May 2024 - Sensor-26

Towards More Accurate 3D Object Detection for Robots
and Self-Driving Cars
Researchers have developed a network that combines 3D LIDAR and 2D image data to
enable a more robust detection of small objects.
Ritsumeikan University, Kyoto, Japan
R
obots and autonomous vehicles can
use 3D point clouds from LIDAR
sensors and camera images to perform
3D object detection. However, current
techniques that combine both types of
data struggle to accurately detect small
objects. Now, researchers from Japan
have developed DPPFA−Net, an innovative
network that overcomes challenges
related to occlusion and noise introduced
by adverse weather.
Robotics and autonomous vehicles
are among the most rapidly growing
domains in the technological landscape,
with the potential to make work
and transportation safer and more efficient.
Since both robots and self-driving
cars need to accurately perceive their
surroundings, 3D object detection
methods are an active area of study.
Most 3D object detection methods employ
LIDAR sensors to create 3D point
clouds of their environment. LIDAR
sensors use laser beams to rapidly scan
and measure the distances of objects
and surfaces around the source.
However, using LIDAR data alone can
lead to errors due to the high sensitivity
of LIDAR to noise, especially in adverse
weather conditions like during rainfall.
To tackle this issue, scientists have
developed multi-modal 3D object detection
methods that combine 3D
LIDAR data with 2D RGB images taken
by standard cameras. While the fusion of
2D images and 3D LIDAR data leads to
more accurate 3D detection results, it
still faces its own set of challenges, with
accurate detection of small objects remaining
difficult. The problem mainly
lies in properly aligning the semantic
information extracted independently
from the 2D and 3D datasets, which is
hard due to issues such as imprecise calibration
or occlusion.
Against this backdrop, a research team
led by Professor Hiroyuki Tomiyama from
Ritsumeikan University, Japan, has developed
an innovative approach to make
multi-modal 3D object detection more accurate
and robust.
26
However, accurately aligning data
Multi-modal 3D object
detectors in
autonomous vehicles
(AVs) and robots
Combine LIDAR and
2D image data to
obtain better results
from different sensors is challenging,
especially for small objects
Complex calibration
Weather conditions
Occlusion
Dynamic Point-Pixel Feature Alignment Network (DPPFA-Net)
for small 3D object detection
Deformable Point-Pixel
Fusion (DPPF) modules
DPPF
Memory-based Point-Pixel
Fusion (MPPF) Modules
MPPF
Intra-modal and cross-modal feature
interactions based on a memory bank
Easier network learning with promotion
of key figures
Robust against noise in LIDAR data
Establish interactions between key
high-resolution features
Reduce computational complexity
Sematic Alignment Evaluator (SEA) modules
SAE
DPPFA-Net achieves a new state-of-the-art for 3D object detection,
paving the way for safer and more capable AVs and robots
The proposed model adopts innovative strategies that enable it to accurately combine 3D LIDAR data
with 2D images, leading to significantly better performance than state-of-the-art models for small
target detection, even under adverse weather conditions. (Image:
Ritsumeikan University)
Hiroyuki Tomiyama from
The model comprises an arrangement
of multiple instances of three novel
modules: the Memory-based Point-Pixel
Fusion (MPPF) module, the Deformable
Point-Pixel Fusion (DPPF) module, and
the Semantic Alignment Evaluator
(SAE) module. The MPPF module is
tasked with performing explicit interactions
between intra-modal features (2D
with 2D and 3D with 3D) and crossmodal
features (2D with 3D). The use of
the 2D image as a memory bank reduces
the difficulty in network learning and
makes the system more robust against
noise in 3D point clouds. Moreover, it
promotes the use of more comprehensive
and discriminative features.
In contrast, the DPPF module performs
interactions only at pixels in key positions,
www.techbriefs.com
which are determined via a smart sampling
strategy. This allows for feature fusion in
high resolution at a low computational
complexity. Finally, the SAE module helps
ensure semantic alignment between both
data representations during the fusion
process, which mitigates the issue of feature
ambiguity.
The researchers tested DPPFA−Net by
comparing it to the top performers for
the widely used KITTI Vision Benchmark.
The proposed network achieved average
precision improvements as high as 7.18%
under different noise conditions. To further
test the capabilities of their model,
the team created a new noisy dataset by
introducing artificial multi-modal noise
in the form of rainfall to the KITTI dataset.
The results show that the proposed
Sensor Technology, May 2024
http://www.techbriefs.com
Tech Briefs Magazine - May 2024

Table of Contents for the Digital Edition of Tech Briefs Magazine - May 2024
