IEEE - Aerospace and Electronic Systems - July 2021 - 74

Artificial Intelligence and Data Fusion at the Edge
huge investments on expensive data centers. Data fusion
and AI at the edge imparts scalability to an intelligent
computing system because majority of the data fusion and
AI computations are performed at edge devices and only
limited fused and processed data need to be sent over the
network to the cloud.
Sustainability. Data fusion and AI at the edge provide
a sustainable solution for emerging smart applications
(e.g., autonomous vehicles, smart agriculture, surveillance,
and swarm intelligence) because increasing advances
in semiconductor will continue to make edge devices
more powerful to carry out inferences in real time in a
cost-effective manner. Moreover, edge AI will be able to
meet AI needs for applications even in communicationdenied
environments or places where no infrastructure
exists for connection to the cloud.
EXPERIMENTAL RESULTS
In this section, we provide experimental results demonstrating
latency, energy, and precision advantages of
combined data fusion and AI at the edge. The experimental
results present latency and energy consumption
comparison between two CNN models with and without
data fusion. Furthermore, results demonstrate how
multimodal fusion help improve the precision of AI
for a CGS system.
EXPERIMENTAL SETUP
We conduct two set of experiments to demonstrate
latency, energy, and precision advantages of combined
data fusion and AI at the edge. The primary set of experiments
use handwritten digit datasets whereas secondary
experiments use different types of camera sensors, viz.,
visual (VI) and medium wavelength infrared (MWIR) outfitted
on UAVs.
In our primary set of experiments, we obtain experimental
results for MNIST handwritten digit dataset 33.We
clarify that we have chosen this dataset for illustrating the
effectiveness of combined data fusion and AI; however,
experimental results for other datasets can be obtained
similarly. For practical relevance, we note that for social
intelligence, often handwritten digits and text need to be
analyzed. We have randomly selected 100 handwritten
images from the dataset for each digit. We have made 10
sets of each digit image where each set contains 10 handwritten
images. We then fuse the images in each set to
produce 10 fused images for each digit. Our data fusion
technique adds the pixel values at the same location of the
10 images in the set. Thus, our primary set of experiments
demonstrates competitive pixel data fusion done explicitly
and model data fusion (knowledge/feature data fusion)
implicitly by the CNN inference.
74
AVERAGE LATENCY FOR COMBINED DATA FUSION
AND AI AT THE EDGE
Table 4 shows the speedup of AI (LeNet and AlexNet
CNN models) with data fusion over AI without data
fusion. Results verify that AI with data fusion provides
significant speedups over AI without data fusion. For
example, AI with data fusion on Xeon CPU results in a
speedup of 2.6 and 9.7 for LeNet and AlexNet,
Table 4.
Speedup of AI With Data Fusion As Compared to AI
Without Data Fusion
Edge Computing
CNN Model
Platform
Xeon_CPU
JTX2_CPU
JTX2_CPU + JTX2_GPU
IEEE A&E SYSTEMS MAGAZINE
LeNet AlexNet
2.55X 9.72X
3.82X 9.76X
9.57X 9.29X
JULY 2021
We consider the following three use cases for AI and
data fusion on multiple hardware platforms suitable for
edge computing:
(1) data fusion and CNN model execution in Intel Xeon
CPU [9] (Xeon_CPU);
(2) data fusion and CNN model execution in Nvidia Jetson
TX2 GPU (JTX2 34) CPU (JTX2_CPU);
(3) data fusion in JTX2 CPU and CNN model execution
in JTX2 GPU (JTX2_CPU+JTX2_GPU).
We average the execution time of performing data
fusion and CNN inference over 10 sets of 10 digits. We
have run our experiments on Ubuntu 18.04 operating system
using CUDA 10.1 as a general purpose GPU framework,
and Python 3.6.9 for implementation of data fusion.
For implementing AI with data fusion, first the 10 images
in each set are fused using data fusion in Python and the
resulting fused image is then provided as input to the Darknet
framework [35] for CNN inference. We use two
CNN models, i.e., LeNet [36] and AlexNet [37], for our
experiments. Regarding the training of CNN models, we
utilize the trained weights provided by Krizhevsky et al.
[38] for LeNet, whereas we train the weights of AlexNet
ourselves using MNIST training dataset comprising of
60 000 images. To smooth out any inconsistencies in
latency due to operating system overhead and variations
in environmental parameters, we average the execution
time results over 10 independent measurements.
IEEE - Aerospace and Electronic Systems - July 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2021
