IEEE - Aerospace and Electronic Systems - July 2021 - 104

On Digital Ethics for Artificial Intelligence and Information Fusion in the Defense Domain
" means, " e.g., by " understood " images. The " labeling " of
training data requires human understanding. If training has
been " long enough, " the network is offered an arbitrary input
and the output is considered the recognized " what, " i.e., the
" meaning " ofthe input. Neural networks are, thus, essentially
function approximators. Whoever calls massive offering of
interpolation points " learning " may awaken erroneous associations
in nonspecialists.
As it turns out, however, only a few pixels in an input
image, for example, need to be changed in a specific way
to completely mislead even well-trained networks. A neural
network, deceived by such " poisonous noise, " may
" misrecognize " a panda bear, for example, which appears
unchanged to humans, as a gibbon monkey and " feels "
certain in its judgment [39]. The military relevance of this
discovery is obvious. Attack systems against AI systems
have already been developed; own AI systems are to be
hardened against such " adversarial attacks. " A situation
occurs as in electronic warfare where electronic measures
call for countermeasures, these for counter-countermeasures
and so on.
In addition, appropriately representative training data
for data-driven algorithms, such as neural networks, are for
most militarily relevant applications unavailable in sufficient
amount. Moreover, such algorithms are often " black
boxes. " Nobody knows how these achieve their results. Furthermore,
context knowledge-fundamental to every military
mission-can only be learned indirectly from the
training data themselves. In short, data-driven algorithms
are " greedy, brittle, and opaque " [40], [41] and always provide
the " second-best solution only " [42]. At least for critical
functions in the targeting cycle, meaningful human
control is required. Model-based algorithms, on the other
hand, allow logical reasoning also in case of uncertainty,
uncover probable cause-effect chains, deliver probabilistic
assessments of the estimates provided, can be developed
systematically, and explicitly allow the integration ofcontext
and expert knowledge. In certain militarily relevant
cases, however, the required models are not available or
too complex to be dealt with efficiently. A still unsolved
problem of current research is the combination of datadriven
and model-based algorithms resulting in explainable
AI.
According to these considerations, the following
issues are pressing in the military domain, but relevant in
other domains as well and need to be addressed by systems
engineering.
1. Responsible use of technology requires consistent
controllability. In some applications, occasional
malfunction ofAI and automation may have no consequences.
In military use, however, rigorous safety
requirements must be guaranteed with all legal consequences.
The military use of technically uncontrollable
technology is immoralper se.
104
2. The notion of " meaningful human control, " on the
other hand, needs to be interpreted more broadly
than the concept of " human-in/on-the-loop " suggests.
Formulations such as: " For unmanned aerial
vehicles, the principle of human-in-the-loop and
thus the immediate possibility of operator intervention
must be ensured at all times " in official documents
of the German Luftwaffe should, thus, be
reconsidered [34, p. 23]. More fundamental is
" accountable responsibility " to be discussed ahead.
The use of fully automated effectors on unmanned
platforms may well be justifiable, even necessary,
in certain situations if appropriately designed.
3. Certification and qualification of AI-based automation
are key issues. Robust military systems will
comprise both data-driven and model-based algorithms,
where data-driven algorithms could be
" contained " by model-based reasoning-AI in the
Box. Predictable system properties, insensitivity to
unknown effects, adaptivity to variable usage contexts,
and graceful degradation must be verified.
Statistical testability as well as explainability is
essential prerequisites for critical components.
Finally yet importantly, compliance to a code of
conduct must be guaranteed by design.
4. Sensor and context data nevermeet ideal expectations.
They are always imperfect, inaccurate, ambiguous,
unresolved, corrupted or deceptive, difficult to be formalized,
or partly contradictory. Statistical models
exploited by powerful algorithms, however, enable
responsible action even on an imperfect data basis. In
many cases, reliable situational awareness pictures
can be inferred from them in a much more precise,
complete and faster way than humans could ever have
hoped to obtain. Nevertheless, also these methods
have their limitations, which decision-makers must
not only be made aware of, but also be interpreted to
them. Under certain circumstances, proper models are
unavailable and have to be learnt provided appropriate
data exist.
5. Data integrity is fundamental to any use ofAI-based
systems: Are valid sensor and context data available
at all? Are they produced reliably and do the
unavoidable deficits correspond to the statistical
assumptions made? In naive systems, violated integrity
easily turns data fusion into confusion. Moreover,
algorithms always generate artifacts that do
not exist in reality, or have " blind spots, " i.e., do not
show what is actually there. In the military context,
enemies may take over sensors or subsystems,
which then produce deceptive data or unwanted
action. Mature AI comprises detection of such deficits,
which is the basis for making own assistance
IEEE A&E SYSTEMS MAGAZINE
JULY 2021

IEEE - Aerospace and Electronic Systems - July 2021

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - July 2021