IEEE Geoscience and Remote Sensing Magazine - March 2022 - 104

network to learn discriminative deep feature representations
for MOT.
Leal-Taixé et al. [180] developed a multimodal MOT
method by learning the local features of red, green, blue
(RGB) images and optical flow maps using a Siamese network.
Al-Shakarji et al. [148] designed a time-efficient
detection-based multiobject tracking system using a threestep
cascaded data association scheme. Dike and Zhou
[162] proposed a quadruplet network to track prediction
objects from crowded environments. Yu et al. [157] proposed
a self-balance method integrating the appearance
similarity and motion consistency. Youssef and Elshenawy
[80] achieved MOT by cascade region-based CNNs and feature
pyramid networks.
It should be noted that, if we directly use video data acquired
by a UAV during the flight for MOT, the detection
result often contain high noise, false alarms, and missed detection
due to changes in the
SIMILAR TO THE VID, MOT
MAY JUDGE A NEW OBJECT
STATUS THROUGH
HISTORICAL TRAJECTORY
INFORMATION.
UAV's motion, the inevitable
" jitter, " and ambient light.
Therefore, it is necessary to
preprocess the UAV video. In
addition, TBD would fail to efficiently
match when the front
and back frames of the object
in the video move too fast.
SINGLE OBJECT TRACKING-ASSISTED
MULTIPLE OBJECT TRACKING
Trajectory prediction can address the identified failings of
TBD well, and the most commonly used technique is the
SOT-assisted method [153], [165], [182]-[185]. With significant
progress in this approach recently, SOT has been
successfully applied to complex scenes [186]-[188], but directly
applying SOT to MOT would encounter calculation
inefficiency and tracking drift caused by occlusion. For this
reason, Pan et al. [153] propose a hierarchical multitarget
tracker (HMTT) incorporating SOT and Kalman filtering to
improve the MOT performance. Li et al. [182] designed a
multiple vehicle tracking approach to effectively integrate
SOT-based forward position prediction with IOUT to enhance
the detection results in the association phase. Yan
et al. [183] associated detector and SOT trackers as candidate
objects, and then candidates were selected through an
ensemble framework.
Xiang et al. [165] adopted the MDP method to track
objects in a tracked state with optical flow. Chu et al. [184]
treated all of the detection output as SOT proposals and
designed an MOT network architecture by considering
multiple objective interactions, yielding a significant improvement
for MOT. A novel instance-aware tracker was
proposed in [189] to effectively integrate SOT into MOT.
In [190], the authors adopted a Siamese RPN [191] SOT
tracker and reidentification (ReID) network to extract
short- and long-term clues, respectively. A better data association
method called switcher-aware classification was
104
then proposed to improve the tracking results while solving
the offset problem.
In these methods, the SOT tracker is independent of data
association, which raises a potential issue that the two steps
do not collaborate well to reinforce each other. To this end,
Zhu et al. [192] proposed dual matching attention networks
to deal with intraclass distractors and frequent interactions
between objects through integrating a unified framework by
single object ECO tracking and data association. In addition,
for the real-time analysis of the SOT-assisted method, offline-trained
SOT trackers, like the Siamese RPN, can achieve
a high-speed accuracy of more than 80 frames/s, while an online
SOT update consumes a lot of CPU resources.
MULTIPLE OBJECT TRACKING BASED ON
MEMORY NETWORKS
Similar to the VID, MOT may judge a new object status
through historical trajectory information. Therefore, it is a
feasible framework for designing a network structure that
can memorize historical information and learn matching
similarity measurement based on it to enhance the performance
of MOT [193]. Among all of the RNNs, the LSTM
network has shown reliable performance on many sequence
problems and can overcome the gradient disappearance and
explosion problems of standard RNNs. The special structures
of LSTM enable it to remember information for a long time.
Recently, some methods [194]-[196] have achieved impressive
performances by employing LSTM networks. Milan
et al. [193] trained an end-to-end LSTM network for
online MOT. Sadeghian et al. [185] integrated appearance,
action, and interaction cues into a unified RNN and designed
the feature fusion based on LSTM to express motion
interaction so as to learn the matching similarity between
the trajectory history information and current detection.
After designing and analyzing each gate function in
LSTM, Kim et al. [197] proposed a novel RNN model
called bilinear LSTM based on multiplication so as to improve
the learning ability of long-term appearance models.
Yu et al. [159] estimated the individual motion and global
motion by LSTM and a Siamese network. In [190], the
short-term clues obtained by the Siamese RPN network
and long-term ones obtained by ReID were introduced to
meet complex scenarios and achieved a state-of-the-art
tracking performance.
MULTIPLE OBJECT TRACKING BASED
ON OTHER APPROACHES
Besides the aforementioned methods, other techniques for
MOT are also available, such as generalized graph differences
(GGDs) for network flow optimization [155] with
an efficient representation of variations between graphs, a
context-aware intersection over union (IoU)-guided tracker
(COMET) [156] with offline proposal generation, and a
multitask two-stream network. There is also literature focusing
on designing MOT patrol [149] or mobile [117] systems
for UAV video.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE MARCH 2022
IEEE Geoscience and Remote Sensing Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - March 2022
