IEEE Systems, Man and Cybernetics Magazine - July 2019 - 58

those AI methods more transparis a prerequisite for trust in this
ent and by educating common peocontext (Figure 6). Historically,
We would strongly
ple through different media.
people used machines over which
argue that creating
they had total control. Whenever
Ethical and Legal Matters
they lost control over a machine,
intention-aware
Autonomous systems raise some
it was, in effect, malfunctioning.
autonomous
technological and social challengA long history of these events
es in terms of reliability, confidencreated a perception of tr ust
systems is key to the
tiality, integrity, and availability.
through control. Since we are
upcoming integration
These characteristics have a direct
operating in this environment, we
impact on the trust dynamics in
must honor this perception and
of autonomous
autonomous systems. Challenges
allow the control of any autonosystems into society.
include ethical and legal implicamous system to be overridden at
tions that depend on the applicathe will of the interacting human.
tion and domain. For instance,
An absence of such control will
advances in autonomous systems
eliminate the possibility of creathave created a new challenge in obtaining informed coning trust between humans and autonomous systems.
sent from patients. Informed consent is the basic element
in patient-surgeon relationships. It helps the patient to
Ensuring Transparency and Predictability
make an educated decision about undergoing a procedure
Is Crucial in Human-Machine Trust
by exercising free will. As autonomous systems become
In today's world, nobody loves a surprise when a critical
more complex, it becomes more difficult to explain them
function is performed by a machine. Due to existing perto patients and their families in a comprehensive form
ceptions and our exposure to different forms of machines,
that includes all relevant details. Also, health-care practiwe are accustomed to seeing them follow a strict routine.
tioners need to deal with the unsuitability of the old conFor example, we know that when a car, which is a highly
sent forms for the new technologies. Using virtual reality
automated machine, is turning right, it is performing a
to demonstrate the process of the procedure in a simulatvery strict series of actions. Our trust in a car perfectly
ed environment can help to achieve a more comprehenturning is mostly because we know the sequence of
sive informed consent.
actions, even from a layperson's point of view (Figure 7).
The concepts behind negligence prosecution in tradiHowever, the technologies used in recent autonomous systiona l med ici ne ca n be a lso applied i n the ca se of
tems are mostly based on deep learning and complex AI
autonomous robotics, as these concepts are related to the
algorithms. They are, very often, black-box methods with
standard of care. Negligence in this context is based on
little to no explanation available. Since we are accusfour principles: noncompliance to duty of care, deviation
tomed to routines for machines, which in turn ensure prefrom standards of care, lesion evidence (in detection/
dictability and transparency for them, we must also make
diagnostic procedures), and causal link between the
sure that the actions of any autonomous system are translesion and the lack of care. For legal implications to apply
parent and predictable. This can be achieved by making

Figure 7. We trust a car doing its job because we
Figure 6. Control is a necessary condition for trust of

any form of machine. (Source: Pixabay.)

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Ju ly 2019

know the series of actions it takes in response to a
command from a driver. This ensures transparency
and predictability, which enable us to trust that the
car is driving properly. (Source: Pixabay.)

IEEE Systems, Man and Cybernetics Magazine - July 2019

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - July 2019