IEEE Technology and Society Magazine - Winter 2014 - 24
artificial agent developers, have recognized this principle. For example, in their paper on responsible agent
behavior, Mamdani and Pitt [17] explain that "...the
interaction between agent and owner must be tightly
coupled in the sense of an identifiable human taking
responsibility for an agent's behavior." "Tight coupling," they write, "does not imply that an agent's
every action needs the owner's sanction, but that the
owner assumes responsibility for all possible behavior
of an agent while it acts autonomously as its owner's
surrogate" [17, p. 28].
Keeping this principle front and center goes hand
in hand with recognizing the contingency of technological development and resisting the presumption
that technological development is merely an inevitable
outcome of what nature allows. What developing technologies will come to look like in the future, how they
will operate, how they will be used, and who will be
responsible for them are all matters contingent upon
negotiations among many different actors [5]. Engineers, researchers, and users have a strong influence
on technological developments, as do financiers, lawyers, policy makers, the media, and the public. What
gets designed, adopted, and used is, thus, the result of
many human actors making decisions that accumulate
to produce an artifact as well as a set of social practices
that together constitute a sociotechnical system.
Recommendation 1:
Artificial agents should be understood to be sociotechnical systems or networks consisting of artifacts
and social practices organized to accomplish specified
tasks through their interactions.
We can understand artificial agents to be merely
artifacts (software), but to do so is to abstract the
artifactual component from the system in which it is
embedded. Without the system the software has no
functionality, no meaning, no effect. Consider, for
example, that a bot is nothing without the Internet, the
operators that maintain the Internet, users who receive
the results of bot operations, and so on. Or consider
an autonomous car. The physical car functions as part
of a system involving other artifacts (e.g., roads, stop
lights, gas pumps) and humans. Autonomous cars
come into being because of human decision-making
and behavior, and they operate only in combination
with human behavior. Humans design and build the
cars and put them on the road. During the development phase, humans monitor the behavior of the cars,
making adjustments as needed. Humans decide when
the autonomous cars are ready for road testing and
humans will decide when they are safe enough for
commercial release. Moreover, how autonomous cars
operate once they are introduced on public roads is
as much dependent on their many different technical
components, as it is on a variety of social practices
24
|
that ensure that the cars are operational as well as safe.
Yes, intelligent algorithms, sensors, and vast amounts
of data enable the car to drive without a human driver
manually controlling it, but the car's ability to navigate the roads also depends on a wide range of human
behavior including legal and social norms that have
been put in place through social processes and inculcated in human drivers. In order to understand why an
autonomous car does what it does or when it fails to
function as expected, we have to look at the car as a
sociotechnical system.
Failure to recognize artificial agents as sociotechnical systems renders the role of human actors in technological outcomes invisible; it invites the idea that no
humans are responsible for those outcomes. Taking a
sociotechnical perspective on artificial agents brings
into view the social as well as technical components
that constitute the technology, so that attention can be
given to how the two work together (see for example
[31, ch. 8], [32]). This perspective facilitates developers and engineers as well as policy-makers and
managers in seeing the multiple actors involved in
artificial agent development and operation. It facilitates recognition of the interactions between humans
and agents and the different levels of responsibility.
It helps to anticipate uncertainties in the development
and implementation process.
To illustrate how a sociotechnical perspective can
help in reasoning through how a new technology
can affect responsibility in a network of human and
non-human actors, think of a healthcare robot that
can autonomously find its way around a hospital to
deliver telemedicine care [1], [27]. Such robots operate alongside and in interaction with hospital employees, patients, and visitors. Of course, developers and
engineers have a responsibility to ensure that the
technology operates according to its technical specifications and that it does not pose any danger to the
people interacting with it. At the same time, users and
operators have a responsibility to use the technology
as it is intended to be used. A sociotechnical perspective, however, also draws attention to the responsibilities of the other human actors that are involved in
the development and operation of these robotic systems. For instance, engineers on the design team were
responsible for making sure that the technology was
adequately tested; the hospital is responsible for training those who operate and maintain the robot; during
the purchase process, hospital representatives had to
make sure the robot would not operate in a way that
conflicts with other organizational procedures and
practices within the hospital; and so on.
Recommendation 1 is not just in harmony with
our overarching principle; it supports recognition of
the principle. When those involved fail to see that
artificial agents are sociotechnical systems, they are
IEEE TECHNOLOGY AND SOCIETY MAGAZINE
|
WINTER 2014
Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - Winter 2014
IEEE Technology and Society Magazine - Winter 2014 - Cover1
IEEE Technology and Society Magazine - Winter 2014 - Cover2
IEEE Technology and Society Magazine - Winter 2014 - 1
IEEE Technology and Society Magazine - Winter 2014 - 2
IEEE Technology and Society Magazine - Winter 2014 - 3
IEEE Technology and Society Magazine - Winter 2014 - 4
IEEE Technology and Society Magazine - Winter 2014 - 5
IEEE Technology and Society Magazine - Winter 2014 - 6
IEEE Technology and Society Magazine - Winter 2014 - 7
IEEE Technology and Society Magazine - Winter 2014 - 8
IEEE Technology and Society Magazine - Winter 2014 - 9
IEEE Technology and Society Magazine - Winter 2014 - 10
IEEE Technology and Society Magazine - Winter 2014 - 11
IEEE Technology and Society Magazine - Winter 2014 - 12
IEEE Technology and Society Magazine - Winter 2014 - 13
IEEE Technology and Society Magazine - Winter 2014 - 14
IEEE Technology and Society Magazine - Winter 2014 - 15
IEEE Technology and Society Magazine - Winter 2014 - 16
IEEE Technology and Society Magazine - Winter 2014 - 17
IEEE Technology and Society Magazine - Winter 2014 - 18
IEEE Technology and Society Magazine - Winter 2014 - 19
IEEE Technology and Society Magazine - Winter 2014 - 20
IEEE Technology and Society Magazine - Winter 2014 - 21
IEEE Technology and Society Magazine - Winter 2014 - 22
IEEE Technology and Society Magazine - Winter 2014 - 23
IEEE Technology and Society Magazine - Winter 2014 - 24
IEEE Technology and Society Magazine - Winter 2014 - 25
IEEE Technology and Society Magazine - Winter 2014 - 26
IEEE Technology and Society Magazine - Winter 2014 - 27
IEEE Technology and Society Magazine - Winter 2014 - 28
IEEE Technology and Society Magazine - Winter 2014 - 29
IEEE Technology and Society Magazine - Winter 2014 - 30
IEEE Technology and Society Magazine - Winter 2014 - 31
IEEE Technology and Society Magazine - Winter 2014 - 32
IEEE Technology and Society Magazine - Winter 2014 - 33
IEEE Technology and Society Magazine - Winter 2014 - 34
IEEE Technology and Society Magazine - Winter 2014 - 35
IEEE Technology and Society Magazine - Winter 2014 - 36
IEEE Technology and Society Magazine - Winter 2014 - 37
IEEE Technology and Society Magazine - Winter 2014 - 38
IEEE Technology and Society Magazine - Winter 2014 - 39
IEEE Technology and Society Magazine - Winter 2014 - 40
IEEE Technology and Society Magazine - Winter 2014 - 41
IEEE Technology and Society Magazine - Winter 2014 - 42
IEEE Technology and Society Magazine - Winter 2014 - 43
IEEE Technology and Society Magazine - Winter 2014 - 44
IEEE Technology and Society Magazine - Winter 2014 - 45
IEEE Technology and Society Magazine - Winter 2014 - 46
IEEE Technology and Society Magazine - Winter 2014 - 47
IEEE Technology and Society Magazine - Winter 2014 - 48
IEEE Technology and Society Magazine - Winter 2014 - 49
IEEE Technology and Society Magazine - Winter 2014 - 50
IEEE Technology and Society Magazine - Winter 2014 - 51
IEEE Technology and Society Magazine - Winter 2014 - 52
IEEE Technology and Society Magazine - Winter 2014 - 53
IEEE Technology and Society Magazine - Winter 2014 - 54
IEEE Technology and Society Magazine - Winter 2014 - 55
IEEE Technology and Society Magazine - Winter 2014 - 56
IEEE Technology and Society Magazine - Winter 2014 - 57
IEEE Technology and Society Magazine - Winter 2014 - 58
IEEE Technology and Society Magazine - Winter 2014 - 59
IEEE Technology and Society Magazine - Winter 2014 - 60
IEEE Technology and Society Magazine - Winter 2014 - 61
IEEE Technology and Society Magazine - Winter 2014 - 62
IEEE Technology and Society Magazine - Winter 2014 - 63
IEEE Technology and Society Magazine - Winter 2014 - 64
IEEE Technology and Society Magazine - Winter 2014 - 65
IEEE Technology and Society Magazine - Winter 2014 - 66
IEEE Technology and Society Magazine - Winter 2014 - 67
IEEE Technology and Society Magazine - Winter 2014 - 68
IEEE Technology and Society Magazine - Winter 2014 - 69
IEEE Technology and Society Magazine - Winter 2014 - 70
IEEE Technology and Society Magazine - Winter 2014 - 71
IEEE Technology and Society Magazine - Winter 2014 - 72
IEEE Technology and Society Magazine - Winter 2014 - 73
IEEE Technology and Society Magazine - Winter 2014 - 74
IEEE Technology and Society Magazine - Winter 2014 - 75
IEEE Technology and Society Magazine - Winter 2014 - 76
IEEE Technology and Society Magazine - Winter 2014 - 77
IEEE Technology and Society Magazine - Winter 2014 - 78
IEEE Technology and Society Magazine - Winter 2014 - 79
IEEE Technology and Society Magazine - Winter 2014 - 80
IEEE Technology and Society Magazine - Winter 2014 - Cover3
IEEE Technology and Society Magazine - Winter 2014 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2023
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2022
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2021
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2020
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2019
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_december2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_september2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_june2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_march2018
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2017
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2016
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2015
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2014
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_winter2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_fall2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_summer2013
https://www.nxtbook.com/nxtbooks/ieee/technologysociety_spring2013
https://www.nxtbookmedia.com