IEEE Robotics & Automation Magazine - March 2011 - 44

(a)

(b)
...?

E = mc2

(c)
Figure 4. (a) Sharkey argues that the cognitive capabilities of
robots do not match with that of humans, and thus lethal robots
are unethical, as they may make mistakes more easily than
humans [85]. (b) Arkin believes that although an unmanned system
will not be able to perfectly behave in battlefield, it can perform
more ethically than humans [9]. (c) In part, the question about the
morality of using robots in the battlefield involves commitments on
the capability of artificial intelligence. (Photo courtesy of the
soldier's silhouette by Ruminglass and Quibik.)

necessity [14]. It is clear that the margin that separates us
from having fully autonomous-armed systems in the battlefield is thinning. Even if all armed robots were to be
supervised by humans, one may still wonder to what extent
the human is still in control [9]. Moreover, there may be
cases where one cannot avoid giving full autonomy to the
system. For instance, combat aircrafts must be fully autonomous to effectively operate [99]. Sharkey predicts that, as
the number of robots in operation in the battlefield increases,
they may outnumber human soldiers. He then argues that
it will become impossible for humans to simultaneously
operate all these robots. Robots will then have to be fully
autonomous [83].
One ethical issue (perhaps the issue that received most
attention to date) arising from increasing autonomy of war
robots has to do with the problem of discriminating
between the fighters and innocent people. This distinction
is at the core of the just war theory [106] and humanitarian
laws [82]. These laws stipulate that only the fighters are
legitimate targets and prohibit attacks against any other
44

IEEE ROBOTICS & AUTOMATION MAGAZINE

MARCH 2011

nonlegitimate targets [84], [14]. Sharkey rightfully argues
that our robots are yet far from having visual capabilities
that may allow to faithfully discriminate between the legitimate and nonlegitimate targets, even in close-contact
encounter [85]. Besides, distinguishing between the legitimate and illegitimate targets is not purely technical and is
further complicated by the lack of a clear definition of what
is a civilian. (The 1944 Geneva Convention advises to use
common sense, and the 1977 Protocol 1 defines a civilian as
any person who is not a fighter [72].) However, even if one
was provided with a precise definition that could be encoded
in a computer program, it is doubtful that robots would
achieve, in a foreseeable future, a level of complexity in robot
cognition that would allow the robot to recognize ambiguous
situations involving a nonlegitimate target manipulating
lethal instruments (such as a situation where a child is carrying guns or ammunition). Sharkey argues that autonomous
lethal systems should not be used, as long as one cannot fully
demonstrate that the systems can faithfully distinguish
between a soldier and civilian, and this in all situations [83].
Lin et al. believe that this is too stringent a condition, since
even humans make errors of this kind (Figure 4) [58]. Arkin
counters that, although unmanned robotic systems may
make mistakes, it would on an average behave more ethically
than human beings [9]. In support of this, Arkin cites the
report from the Surgeon General's Office [96] regarding the
ethics of soldiers. Less than half of the soldiers believed that
the nonfighters should be treated with dignity. The other half
was unclear as to how they should be treated. Moreover, one
tenth of interrogated soldiers had mistreated nonfighters and
one third reported having at least once faced a situation
where they felt incapable of deciding what was the correct
action (although all soldiers had received ethical training).
Since human soldiers appear to misbehave from time to
time, using machines that are more reliable and hence
would, on average, make less mistakes should bring more
good than harm. Lin et al. share the view that human soldiers are indeed less reliable and report on an evidence that
human soldiers may act irrationally when in fear or stress.
Hence, they concur that combat robots, which are affected
by neither fear nor stress, may act more ethically than
human soldiers irrespective of the circumstances [58].
Lin and colleagues point to one more issue related to
using combat robots. As in the case of any other new
technology, errors and bugs will inevitably exist, and these
will lead combat robots to cause harmful accidents [58].
Such bugs or errors will be far more costly as human lives
might be at stake. They advise to perform extensive testing
of each military robot before usage. Nevertheless, they
anticipate that, regardless of such efforts, combat robots
may still occasionally behave in unexpected or unintended
ways when used in the actual field [58]. Such errors could
even lead to accidental wars if the robot's unexpected
aggressive behavior was to be interpreted by the opponent
as an act of war [14]. Groups of people interested in starting
a war may seize upon such accidents to justify hostilities.

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2011