IEEE - Aerospace and Electronic Systems - July 2021 - 56

A Survey of Multimodal Sensor Fusion for Passive RF and EO Information Integration
Table 10.
EO and North sigint Fusion
Scenario
1
2
3
Accuracy
Macro AVG 0.79
Weighted AVG 0.80
Table 11.
EO and West sigint Fusion
Scenario
1
2
3
Accuracy
Macro AVG 0.82
Weighted AVG 0.83
Table 12.
EO and RF sigint Fusion
Scenario
1
2
3
Accuracy
Macro AVG 0.95
Weighted AVG 0.95
0.87
0.67
0.82
scenario categorization. The fusion of EO and all three
P-RF sensors yield the best result.
Besides feature-level fusion, more traditional methods
Precision Recall F1-Score
0.65 0.74
0.92 0.78
0.86 0.84
0.78
0.81 0.79
0.78 0.78
were explored to analyze the effectiveness of using the RF
features and EO input. Logistic regression, random forest,
Naı¨ve Bayes, and GNB were all applied to the same datasets
as the EO-RF fusion NNs. As these ensemble methods
are more traditional in nature and not based on NNs, the
evaluation of classification accuracy is conducted by
k-fold cross validation.
K-fold cross validation is a procedure meant to estiPrecision
Recall F1-Score
0.79
0.92
0.75
0.75 0.77
0.82 0.86
0.92 0.83
0.82
0.83 0.82
0.82 0.82
mate the skill of a machine learning model on unseen
data. The limited samples are used to estimate how the
model is expected to perform in general when used to
make predictions on data that is not used during training.
The process first shuffles the dataset randomly, splitting
up the dataset into k groups. For each of these unique
groups, one is designated as a test dataset, while the others
are treated as part of the training dataset. The model is fit
on the training data and then tested on that dataset, saving
the evaluation score and discarding the model. After
repeating the process, the skill of the model is summarized
using the sample of the model evaluation scores.
As can be seen in Table 13, Logistic Regression and
GNB consistently performed better than Naı¨ve Bayes and
Random Forest in terms of accuracy. The accuracy for these
classifiers improved when there were fewer inputs from the
RF histograms. Considering the nature of the RF features and
the EO inputs, it is possible that because the data received as
a 2-D array that the classifiers that assume the data to be independent,
random forest and Naı¨ve Bayes, performed comparatively
poorer than logistic regression and GNB.
Once the feature-level fusion comparison experiments
Precision Recall F1-Score
0.96
0.99
0.91
0.96 0.96
0.94 0.96
0.96 0.93
0.95
0.95 0.95
0.95 0.95
In order to accurately evaluate the performance of
FERNN, the F1 score, precision, and recall were calculated
based on (1)-(3) for the statistical analysis. The F1
score is the harmonic mean of the precision and recall, the
measurements of positive predictive value and sensitivity
for machine learning. Precision is the measurement of
type-I error, false positives, while recall is the measurement
of type-II error, false negatives. Figure 7 shows the
F1 scores of all the NN trained for vehicle detection and
56
were completed, the architecture for the feature-level
fusion was adopted to implement decision-level fusion.
As seen in Figure 8, at least one source of EO data were
classified with the RF features, and once that classification
Figure 7.
Comparison ofF1 scores for different NNs implemented for vehicle
detection.
IEEE A&E SYSTEMS MAGAZINE
JULY 2021
IEEE - Aerospace and Electronic Systems - July 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2021
