IEEE Robotics & Automation Magazine - March 2023 - 38

Furthermore, we extended the persistence filter with a long
short-term exponential model to make the predicted map more
accurate. The main contributions of this article are as follows:
■ We propose an efficient and accurate segmentation method
to separate static, semistatic, and dynamic map points,
which can eliminate the influence of dynamics in realworld
environments.
■ Our method provides a long-term GMP method with a
map point matching algorithm, which can provide us with
a global map that fits the current environment.
■ An LTVS with a BPF is proposed to improve the robustness
and accuracy of pose estimation in long-term, dynamic
environments. Extensive experiments are carried out to
demonstrate the effectiveness and accuracy of our system.
SYSTEM OVERVIEW
In the following sections, we present a brief introduction to
our system. The architecture of our system is shown in
Figure 2. The white blocks in Figure 2 are the same as in
ORB-SLAM2 [14]. Our system contains several modules,
which are presented as follows.
FRONT- AND BACK-END OPTIMIZATION
In the front-end module, for each incoming stereo/red, green,
blue-depth (RGB-D) frame, ORB features are extracted, and
the global map is initialized. The track observations module is
used to track the status of each map point in the current frame.
When a KeyFrame arrives, the local mapping module is activated
to add map points to the map. Whenever a new map
point is created, the local map update module is activated to
create a new persistence filter and assign it to the map point.
Furthermore, occlusion detection (OD) is employed to avoid a
wrong input to the persistence filter. For each frame arrival, the
persistence probability of all the map points is updated for the
current time stamp. With the persistence filter, the map point
whose persistence probability is under a certain threshold is
considered a dynamic map point and removed from the map.
The KeyFrame and recognition database is then updated in the
local map update module. As for the loop-closing module, it is
used to detect loop closures and refine localization.
TIME-SERIES GLOBAL MAP
Different from the map built by ORB-SLAM, the time-series
global map is built to save the previous observations and
obtain temporal maps of the environment. In our algorithm,
each map Mi consists of the KeyFrames Ki and the map
points Mpi. Each map point Mpi stores the following:
■ 3D position Oi in the world coordinate system.
■ viewing direction ni, which means the ray that joins the
point Mpi with the optical center of the KeyFrames Ki that
observes it.
■ survival-time T [, ),0i 3!
which represents the time when
the map point Mpi ceases to exist.
■ persistence probability of the map point Pi, which is the
output of the persistence filter; it will be used to classify
the map points into static, semistatic, and dynamic.
■ a representative map point descriptor ,Di which is different
from the original ORB descriptor. The specific description
of this descriptor is discussed in the " Semistatic Map
Points Prediction " section.
■ the maximum and minimum observation distance, dmax,
dmin, according to the characteristic of sensor.
OFFLINE TIME-SERIES MODELING
There are three parts in this module: the map points matching,
BPF, and semistatic map points cluster modules. The
map points matching module is used to obtain an accurate
time series of map points. After long-term operation in a
region, the temporal maps of different time tk are constructed
using the front- and back-end optimization modules. The
voxel segmentation is then operated on the temporal maps
to obtain the voxel information of each map point. And then
we fuse the voxel information with the 2D ORB descriptors
and create a new 3D map point descriptor
D .i With this
carefully designed descriptor, the time series of map points
are constructed. Then, with the BPF and survival analyses,
we can model the time series of semistatic map points and
obtain the survival function of the map point. Semistatic
map point clustering is performed to reduce the prediction
time. Hamming and Euclidean distances are used to
describe the distance in temporal and spatial, respectively.
Predicted Map
Stereo/RGB-D
Input With
Dynamic Objects
Visual SLAM
System
History Frames
History Maps With
Dynamic Map
Points Removal
Long-Term
Localization
Intelligent Wheelchair
Current Frames
FIGURE 1. The proposed LTVS system can remove dynamics in environments and predict the future map, resulting in an excellent
performance in localization for intelligent wheelchairs. RDB-D: red, green, blue-depth.
38 IEEE ROBOTICS & AUTOMATION MAGAZINE MARCH 2023
Global Map
Prediction
Application

IEEE Robotics & Automation Magazine - March 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2023