IEEE Robotics & Automation Magazine - March 2023 - 46

LOCALIZATION PERFORMANCE
For real-world long-term deployment, estimation failures or
wrong poses are severe, and they may occur commonly
because of scene changes. Therefore, inspired by [20], we set
correctness ()
cp,
fz
i
and correct rate (CR) to evaluate the correctness
and robustness, respectively, of our system. For each
pose estimate pi at time ti, we obtain the correctness of the
estimate by its ATE and absolute orientation error (AOE):
()
if pp ( ),
fz
wise, () .cp 0
,
ii
i =
fz then () ;cp 1
,
and AOE
i =
iikN0 f=
,,
ATE ## fz
given an estimated trajectory ,, define
N
CR ,fz
N 1 0+
=
/ min ii d $
k=0
+1 -
-
where tt ,max
(( ,) ())
max min
tt
tt cp
fz,
i
(8)
and d is a parameter used to determine
for how long a correct pose estimation is valid. d is larger
than the time cycle of pose estimation and much smaller than
the time span of the sequence. The CR is set to evaluate the
robustness over the whole period of data. The results of localization
performance of our system in different speed-level
datasets are listed in Table 1 and Figure 9. We make comparisons
with some advanced frameworks in real-world datasets
to verify the effectiveness of our system. Our proposed
otherFor
a sequence from tmin to tmax and,
tp
dynamic culling scheme performs better than other methods.
It can better eliminate the influence of those dynamic map
points. Our method also provides a more accurate global map
that fits the current environment well, resulting in better performance
in localization.
To evaluate those modules in the system, we conduct an
ablation study to determine which technology has played a
role in improving localization accuracy. We conduct different
ablation studies: our method without an MPR algorithm,
GMP, OD, and 3D matching. From these results, we
can see that our system (LTVS-BPF) with all those modules
achieves the best localization performance. Every module
contributes to the final performance, and without each module
may decrease the localization accuracy. We can see that
in medium-speed datasets, the dynamic map points module
plays a more important role, and in low-speed datasets,
the GMP module plays a more important role. In mediumspeed
datasets, there are more dynamic objects in the environment,
and the moving speed of these dynamic objects is
relatively great.
In Figure 10 and Table 5, there are results of different
systems' localization performance at different times after
training datasets. In our experiments, the ATE threshold f
is set to 1 m, and the AOE threshold is
30 .c The results in
Table 5 indicate that our system has better correctness and
-2
2
4
6
20
2
10
30
-4
25
25
20
15
-10
-10
10
5
5
y (m)
(a)
-20
-5 -10 -30
5
30
-20
x (m)
10
30
-4
25
25
20
20
15
15
-10
-10
10
y (m)
5
5
(b)
-20
-20
-5 -10 -30
5
-2
2
4
6
10
10
x (m)
20
-2
2
4
6
20
2
30
-4
25 20
-10
-10
15
10
y (m)
(c)
5
5
-20
-5 -10 -30
5
30
-20
10
10
x (m)
-2
2
4
6
30
-4
25
20
15
15
-10
10
y (m)
(d)
FIGURE 8. The original map without map prediction, and the predicted maps of different prediction methods. The red map points are
the error map points which are the missed map points or the false map points compared with the current map. (a) The original map.
(b) LTVS-ARMA. (c) LTVS-FreMen. (d) LTVS-BPF.
46 IEEE ROBOTICS & AUTOMATION MAGAZINE MARCH 2023
5
5
-20
-20
-5 -10 -30
5
30
-10
10
x (m)
20
2
z (m)
z (m)
z (m)
z (m)
IEEE Robotics & Automation Magazine - March 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2023
