IEEE Robotics & Automation Magazine - September 2023 - 31

the overall comparison, the experimental results are comprehensively summarized
according to the six following criteria: measurement precision, data dimensions,
cost, distance range, size, and environmental robustness. Such comparative
experiments would provide relatively quantitative information, when considering
extreme environments such as dusty outdoors, or glassy buildings.
LIDAR AND RADAR SENSORS CONSIDERED
The purpose of comparing sensor performances is to obtain insights on factors to
consider when selecting sensors for various robotic applications. For this purpose,
we considered four types of lidars and two types of radars (Figure 2), which are
commonly used and representative with respect to beam steering technologies
and measurable ranges. These sensors have additionally been widely applied in
robotics [11]-[17]: 3D mechanical lidar (Velodyne VLP-16 and Ouster OS1), 2D
mechanical lidar (RPLIDAR-A2), MEMS lidar (Benewake CE30-D), flash lidar
(HPS-3D160-U), single-chip millimeter-wave radar (TI IWR 1443), and IR-UWB
radar (Novelda X4M03). The sensor specifications are summarized in Tables 1
and 2. As VLP-16 and OS-1 belong to the same type of 3D mechanical lidar, one
of them was considered sufficient for comparative evaluations.
OBJECT DETECTION
SETUP
To evaluate depth sensing ability with respect to the material and thickness of
objects, three different types of rods were prepared: wooden, steel, and acrylic,
as shown in the first row of Figure 3. Each rod had a diameter of 2 cm and
a length of 52 cm for steel, 80 cm for wood, and 93 cm for acrylic. This difference
in length is practical for distinguishing rods in unclear lidar and radar
images. Six lidar and radar sensors measured the distances to the rods of
three different materials at various distances. For experiments with lidar sensors,
four rods were aligned 10 cm apart from each other and located at a specific
distance from the sensors. In addition, three bundles grouped with four
rods were fabricated for a more comprehensive experiment. In the case of
radar sensors, the rods were detected without directional information, which
is not the case with lidar sensors. For more objective results, the experiments
were conducted inside a building with a wide, unobstructed space, and the
sensing area did not contain Styrofoam for fixing the rods.
As the data structures of lidar and radar sensors are different, their measurement
methods were set to be unified to ensure an unbiased comparison. The
lidar scan data were obtained for 20 s and averaged to capture the shape of the
rods. In the case of radar sensors, two images were taken with and without rods
for eight frames. The intensity of the received wave at the rods' position was
calculated as the difference between those of these two images, and the same
method was applied for both radar sensors.
Additionally, we studied the effectiveness of sensors toward detecting various
objects that are commonly found in real life: a chair, fire extinguisher, computer
monitor, acrylic box, and cardboard box, which were randomly arranged
in an indoor space. The experimental results are shown in Figure 4. Four lidars
and one radar were tested to ensure the appropriate recognition of each object
in a specific environment. IR-UWB radar was excluded from the experiment as
it cannot measure direction.
RESULTS
The experimental data of each sensor are displayed in Figure 3 for the material
type and distance from objects. In the case of lidar experiments, the number of
points in a point cloud, especially at the positions of rods, is critically dependent
on the object detection ability, as indicated by the yellow boxes in the figure.
SEPTEMBER 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
31
TABLE 1. Specifications of lidar sensors.
SENSORS
VLP-16 [21]
Ouster OS1
(16 channel) [22]
MECHANISM FOV ( ° )
DETECTION
RANGE (M)
Mechanical 360 × 30 Up to 100
Mechanical 360 × 33.2 0.8-120
RPLIDAR-A2 [23] Mechanical 360
CE30-D [24]
MEMS
HPS-3D160-U [25] Flash
N/A: not applicable.
60 × 4
76 × 32
Up to 18
Up to 30
Up to 12
ANGULAR
RESOLUTION ( ° )
0.1-0.4 (horizontal),
2 (vertical)
0.01 (horizontal),
0.01 (vertical)
0.9
0.2
N/A
DISTANCE
RESOLUTION
(MM)
30
30
0.5
10
N/A
PIXEL
RESOLUTION
-
-
-
320
× 20
160 × 60
POINTS
PER
SECOND COST (US$)
OPERATING
VOLTAGE
(V)
600,000 5,000-6,000 9-18
327,680 5,000-6,000 22-26
8,000
319
192,000 1,399
336,000 269
5
12
9-12
LASER
WAVELENGTH
(NM)
905
850
785
850
850
SCAN
RATE (HZ) WEIGHT (G)
5-20
10/20
10
-
-
830
380
190
334
140
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
