Systems, Man & Cybernetics - January 2017 - 16

software-intensive HOTL RAS become
more prevalent, the threat of attacks
from hackers grows. Cyberattacks on
* Sensing
these systems through the digital net* Understanding
work (hacking) could lead to catastro* Automated Learning
phe s. Vu l ner a bi l it ie s ex ploit ed by
* Perception
cyberattackers could be in the software
* Decision Making
of RAS, in security policies, or in commuand Planning
nication technologies. Poorly designed
* Action
and protected RAS become vulnerable
targets for criminals or extremists to
remotely take partial or full control of
Figure 3. the teaming of future autonomous systems. [Images
courtesy of cgtrader (soldier), Pintrest/caudate-nucleus (brain), and
these systems a nd m isuse them to
stock-clip.com (skeleton).]
achieve their objectives. An attacker can
manipulate all safety-critical systems
through
accessing
a vehicle's electronic control unit [29].
still be imperative not to accidentally sacrifice even one
There is no question or doubt that RAS such as driverhuman life. Therefore, it is not probable that fully autonoless cars could be hijacked. A car hijacked by a malicious
mous RAS will be sent into the battlefield when civilian
party becomes an immediate physical threat. While car
lives might be at risk, at least not until a satisfactory result
manufacturers focus on safety engineering, the security
could be provided in terms of absolutely trustworthy
aspect of software engineering needs to be enhanced.
detection, evaluation, decision making, learning, and performance. New methodologies and metrics must be developed to evaluate distributive control and situational
Better Legislation and Ethical Regulation
awareness. In addition, developing test protocols to supHow and when RAS can make a correct ethical decision is
port on-board management system evaluation and autonoa hot issue. For example, should a driverless car with full
mous shutdown is also required for future HOTL RAS.
autonomy sacrifice its passengers to avoid crashing into a
Furthermore, developing realistic test-bed components
bus or building full of people? Another example is RAS
and environments to support HOTL RAS evaluation is necused in combat. The use of HOTL RAS in the military raisessary. In this respect, driving simulators (e.g., the Univeres many moral and ethical concerns. For example, is it
sal Motion Simulator [28], as shown in Figure 4) are safer
permissible for HOTL RAS to sacrifice one innocent perand more cost-effective tools to evaluate new autonomous
son to help save tens of others? Existing RAS technology
vehicle designs as well as human behavior in a virtual
complies with the laws of war and rules of engagement.
environment [31]-[33]. The degree of human trust in a
However, the artificial intelligence and robotics societies
designed autonomous vehicle can be measured through
face an important ethical decision: whether to support or
driving simulators before an actual test drive, as can the
oppose the development of lethal autonomous weapons
efficiency of human-robot interaction when the driver is
systems (LAWS) [30]. LAWS is defined as the third revolusupervising the driving scenarios.
tion in warfare, after gunpowder and nuclear arms.
Programming and giving license to HOTL RAS to harm or
kill
under certain conditions are something many experts
Better Protection
have
warned about. How to ensure that humans and RAS
The majority of HOTL RAS, such as driverless cars or
coexist safely is a great challenge. Another is how HOTL RAS
unmanned aerial vehicles, greatly rely on sensors, oncould distinguish between red and blue teams or civilians. If
board computers, and networks to collect, store,
HOTL RAS are not able to tell the difference, many innocent
proce s s, a nd com mu n ic at e d at a . T herefore, a s
people could be hurt or killed. Some environments, including the battlefield, have
more complex social rules than those in factories or hospitals. Therefore, HOTL RAS
should be able to correctly perceive, create,
and understand dynamic models of their surroundings. RAS should accurately categorize
objects around them and recognize and
detect humans as well as their emotions
before making any decision. How humans
and the RAS could complement each other
is also yet to be fully investigated. Due to
these obstacles and the current technology
Figure 4. the Universal motion Simulator [28].
Autonomous
Robot

IEEE SyStEmS, man, & CybErnEtICS magazInE Janu ar y 2017

http://www.stock-clip.com

Table of Contents for the Digital Edition of Systems, Man & Cybernetics - January 2017