IEEE - Aerospace and Electronic Systems - July 2021 - 2

In This Issue -Technically
MACHINE LEARNING SUPPORT FOR RADAR-BASED SURVEILLANCE SYSTEMS
Radar-based surveillance systems already consist ofhighly complex tracking, sensor data fusion, and identification algorithms,
which track the trajectories ofmoving objects. They are embedded in a real-time middleware with a straight forward
processing chain according to the Joint Directors of Laboratories (JDL) fusion model. With the spread of new
technologies, e.g., big data, distributed data processing and machine learning open up new possibilities for surveillance
systems. Commercial data providers provide trajectories of all kinds of vessels and aircraft worldwide. Best known are
automatic dependent surveillance-broadcast and (satellite-) automatic identification systems used in air and maritime surveillance.
Both are cooperative systems and are integrated as the sensor source in surveillance systems. An advantage of
these trajectories is, in addition to the unique identification of the object by an identifier [e.g., ICAO code or MMSI] that
can be easily assigned to the generating objects, they contain additional context data that can be used as labels for supervised
machine learning. Also they are similar in structure to radar tracks and are ideal for analysis and training oflearning
algorithms. This article gives an overview ofhow these new technologies, in combination with big data oftrajectories, can
be integrated into existing surveillance systems and howmachine learning can help to improve situational awareness.
A SURVEY OF ARTIFICIAL INTELLIGENCE APPROACHES FOR TARGET SURVEILLANCE
WITH RADAR SENSORS
With the rising popularity of artificial intelligence (AI), target surveillance based on radar sensors aims to tap the
potential of AI enabled through today's computational capacities. We present a survey of past approaches as well as
recent hot topics in the area of AI approaches for target surveillance with radar sensors that reveal potential for the
development of novel approaches in research and practice. We focus on the major research streams of clutter identification,
target classification, and target tracking, which are important for an adequate operation of radar applications
and well suited for the use of AI. This article contributes to a better understanding of how AI can be applied to assist
conventional radar sensor approaches, or even serve as an alternative.
A SURVEY OF MULTIMODAL SENSOR FUSION FOR PASSIVE RF AND EO INFORMATION INTEGRATION
Integrating information collected by different types of sensors observing the same or related phenomenon can lead to
more accurate and robust decision making. This article reviews sensor fusion approaches to achieve passive radio frequency
(RF) and electro-optical (EO) sensor fusion and presents the proposed fusion of EO/RF neural network
(FERNN). While research has been conducted to integrate complementary data collected by EO and RF modalities,
the processing of RF data usually applies traditional features, such as Doppler. This article explores the viability of
using the histogram of I/Q (in-phase and quadrature) data for the purposes of augmenting the detection accuracy that
EO input alone is incapable of achieving. Specifically, by processing the histogram of I/Q data via deep learning and
enhancing feature input for neural network fusion. Using the simulated data from the Digital Imaging and Remote
Sensing Image Generation dataset, FERNN can achieve 95% accuracy in vehicle detection and scenario categorization,
which is a 23% improvement over the accuracy achieved by a stand-alone EO sensor.
ARTIFICIAL INTELLIGENCE AND DATA FUSION AT THE EDGE
Artificial intelligence (AI), owing to recent breakthroughs in deep learning, has revolutionized applications and services in
almost all technology domains including aerospace. AI and deep learning rely on huge amounts of training data that are
mostly generated at the network edge by Internet ofThings (IoT) devices and sensors. Bringing the sensed data from the
edge ofa distributed network to a centralized cloud is often infeasible because ofthe massive data volume, limited network
bandwidth, and real-time application constraints. Consequently, there is a desire to push AI frontiers to the network edge
toward utilizing the enormous amount ofdata generated by IoT devices near the data source. The merger ofedge computing
and AI has engendered a new discipline, that is, AIat the edge or edge intelligence. To help AI make sense ofgigantic
data at the network edge, datafusion is ofparamount significance and goes hand in hand with AI. This article focuses on
data fusion and AI at the edge. In this article, we propose a framework for data fusion and AI processing at the edge. We
then provide a comparative discussion ofdifferent data fusion and AI models and architectures. We discuss multiple levels
offusion and different types ofAI, and how different types ofAI align with different levels offusion. We then highlight
the benefits ofcombining data fusion with AI at the edge. The methods ofAI and data fusion at the edge detailed in this article
are applicable to many application domains including aerospace systems. We evaluate the effectiveness ofcombined
data fusion and AI at the edge using convolutional neural network models and multiple hardware platforms suitable for
edge computing. Experimental results reveal that combining AI with data fusion can impart a speedup of9.8 times, while
reducing energy consumption up to 88.5% over AI without data fusion. Furthermore, results demonstrate that data fusion
either maintains or improves the accuracy of AI in most cases. For our experiments, data fusion imparts a maximum
improvement of15.8% in accuracy to AI.
2
IEEE A&E SYSTEMS MAGAZINE
JULY 2021

IEEE - Aerospace and Electronic Systems - July 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2021