IEEE - Aerospace and Electronic Systems - July 2021 - 49

Vakil et al.
Table 4.
Methods of Achieving Detection and Tracking via EO/Passive RF Sensor Fusion
Input Data
Method
Passive Radar and EO/
IR sensor input
Unmanned Aircraft Vehicle sense and avoid application using
SVM classifier
FMV and Passive RF Sheaf-based heterogeneous sensor fusion using passive RF
collected via Doppler Radar and FMV for target detection and
tracking.
FMV and Passive RF Joint Manifold Learning based heterogeneous data fusion
approach to form a joint sensor data manifold for vehicle
detection and tracking.
FMV and Passive RF Deep learning approach using feature manifold representations
for multiobject tracking and detection.
FMV and Passive RF Autoencoder based Dynamic Deep Directional-unit network to
achieve unsupervised upstream sensor fusion for the detection
and tracking of vehicles
with the construction and usage of active radar, and are
harder to implement countermeasures against, such as
jamming and spoofing which can corrupt the collection of
RF-based modalities and transmitted imagery. Combining
the EO/RF modalities improves the overall reliability and
has been implemented in a few applications for target
detection, estimation, and tracking, as summarized in
Table 4.
For the research by Barott et al. [6], a SVM is used as
a final method ofclassification. Similar to previously mentioned
papers, Fasano [10] and Kemkemian and Nouvel
[26], which use different metrics for accuracy, they share
the end goal is for the sense-and-avoid of unmanned aircraft.
The purpose of the fusion is to use two complementary
instruments, passive radar, and an EO/IR system to
not only the detection of aircraft, but also the identification
of the model and relative threat to the unmanned aircraft.
The architecture for fusion first preprocesses the thermal
and visible images, isolating the propulsion and aircraft
before extracting the characteristics. These features are
then correlated with the relative distance and orientation
of the radar return, and subsequently to create a multispectral
aircraft signature, which is used as an input for the
SVM classifier.
The use of an autoencoder-based dynamic deep directional-unit
network [20] was capable of learning compact,
abstract feature representations from the high-dimensional
spatiotemporal data of full motion video, and I/Q data.
The architecture exploits the access to elements of interest
within regions of interest using temporal tracking and
supervised classification before being fed into a decentralized
supervised discrimination layer that applies Bayesian
program learning in order to implement upstream multimodal
data fusion. Among the network's achievements, a
notable benefit of the approach is that the network is
JULY 2021
References
[6]
[38]
[41]
[42]
[20]
capable of reconstructing missing modalities given the
observed signatures.
Other research into achieving EO/RF fusion for vehicle
tracking and detection using FMV and P-RF include
joint manifold learning [43],sheaf-based approach with its
data [21], SVM classifier [6]. In [43] and [21], the use of
simulation data is used for the primary method of training
and testing, while in Barott et al. [6] real data collected
from Daytona Beach International Airport are used. In
[43], the use of a joint manifold learning fusion approach
is used for the mixed simulation data. The use of a digital
imaging and remote sensing image generation (DIRSIG)
dataset provides video measurements and three distributed
RF sensors. The intrinsic low-dimensional data, the 2-D
images of the vehicles, are extracted by manifold learning
algorithms from high-dimensional data by implementing a
linear transformation of the vehicle positions. The RF data
are similarly handled by manifold learning, and then the
implementation of linear regression is used for tracking.
These results were compared with a number of methods,
such as maximally collapsing metric learning or neighborhood
preserving embedding, calculating position errors
with respect to the ground truth after implementing noise.
Finally, inRobinson et al. [21], the use ofsimulatedmultisensor
data is used to locate a moving emitter. The method
offusion implemented is SheafTheory, a tool for systematically
tracking locally defined data attached to the open sets
ofa topological set. For the purposes ofimplementing sensor
fusion, the data samples and model of data are used as the
inputs ofa sheaf-based fusion architecture. The model ofthe
data is used to construct the sheafwhile the data samples are
converted into samples for partial assignment. The outputs
of the two are then used to search over the global sections
using the optimizer, before using the results to report values
over the stalks. During testing, the observed stalks for each
IEEE A&E SYSTEMS MAGAZINE
49
IEEE - Aerospace and Electronic Systems - July 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2021
