IEEE - Aerospace and Electronic Systems - July 2021 - 86

Machine Learning/Artificial Intelligence for Sensor Data Fusion-Opportunities and Challenges
fusion strategies to support the user. To realize these benefits,
a human-machine coordination can manage heterogeneous
and uncertain knowledge sources, detect duplicate or incomplete
concepts, and support explainability. One example is the
AI data hierarchy ofneeds, shown in Figure 6 [16].
CHALLENGES AND OPPORTUNITIES
Figure 6.
AI/ML data hierarchy of needs.
Future issues for the coordination of AI/ML and SDF
include:
Common terminology for systems of systems
integration;
Ethical standards for ROE;
Mathematical representations to combine AI/ML
object assessment with SDF situation assessment; and
Coordination of data- and model-driven approaches
for xAI/ML.
MACHINE-HUMAN AI/ML DATA FUSION CONCERNS
Erik Blasch presented emerging issues for the coordination of
AI with SDF including: data preparation for learning, joint
learning and estimation, and user coordination [15]. While
SDF systems typically work with low-dimensional data for
estimation (e.g., target tracking), new methods ofAI/ML seek
high-dimensional analysis (e.g., multi-intelligence graphical
methods). The merging ofAI/ML with SDF requires mathematical
rigor to support thejoint analysis which relies on SDF
methods of data association, uncertainty reduction, and prediction.
Finally, as AI is learning patterns, there is a need to
incorporate physics-based first principle models to provide
structure, explainability, and repeatability.
EXAMPLE: DATA PROCESSING
The SDF problem can leverage AI/ML coordinated with SDF
for: (1) knowledge reasoning and understanding, (2) information
fusion enhancement, (3) object recognition and tracking,
(4) combining data with models, and (5) deep multimodal
To implement AI/ML systems, the first challenge is data
preparation (and aligned with data assessment) involving
collecting, moving, transforming, and labeling data before
AI/ML systems conduct analytics (see Figure 6).
The second challenge, as shown in Figure 7, is testing
AI/ML with SDF systems. With the ability to process big
data (volume, velocity, veracity, variety, value), the collaboration
policies between SDF and AI/ML need to be
understood through evaluation that considers operational
deployment needs such as computation efficiency, adversarial
robustness, and system maintainability. Methods of
robust evaluation for verification and validation support
consistency, reproducible results, and user expectations.
SDF in MDO affords the ability to observe the scenario
from many perspectives, build situation assessment, and learn
patterns. Methods have been shown for AI/ML multi-modal
sensing and image fusion to support object assessment for filtering,
estimation, and prediction [17]. However, AI/ML
needs to coordinate with SDF that brings together systems
issues for command and control (e.g., process refinement)
using dynamic context toward awareness (e.g., user refinement).
Together, SDFandAI/ML will provide enhanced capabilities
from which policies (e.g., guidelines, mandates, and
standards) will dictate the appropriate use ofthe technology.
With the many advantages of AI/ML and SDF, additional
research needed includes:
Simulated (synthetic-measured) data complemented
with data collections for predicting future needs;
PHIF [18] toward needs-based data processing;
Contextual awareness to support AI/ML that leverages
a priori knowledge;
Transfer Learning over operating spaces (sensors,
environments, targets) for joint data association; and
Joint AI/ML/SDF architectures to transcend transparency
and explainability oflearned knowledge.
Figure 7.
Testing and evaluation ofAI/ML system.
86
IEEE A&E SYSTEMS MAGAZINE
JULY 2021

IEEE - Aerospace and Electronic Systems - July 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2021