IEEE Technology and Society Magazine - September 2015 - 63

most irresponsible and most venal
of our engineers" [6].
As Wiener well knew, a technology can go awry even when deployed
by the most responsible and least
venal of engineers, simply because
once unleashed, it interacts with its
environment in ways that are impossible to fully predict or control. This
lack of control is what the often
misunderstood term "technological
autonomy" refers to. A related term
describes the surprising results
our technologies so often produce:
"unintended consequences."
The dream of mastery can be
interrupted by any number of factors, among them human and technical fallibility and chance. Winner
provides a number of quotations
that demonstrate how long and how
often these conditions have been
recognized. One is a poem from
Solon of Athens, dating from approximately the sixth century BCE.
"Danger, for all, lies in all action,
and there is no telling which way
the end will be after a thing is
begun" [6, p. 93].
It's likely that one of the classic
narratives in the counter-tradition to
the dream of mastery, the Tower of
Babel story in the Book of Genesis,
was composed within a century or
so of Solon's poem.
Winner points out that one of the
chief reasons mastery is unattainable is the interconnectedness of
the world. Because of that interconnectedness, repercussions expand
promiscuously; consequences fall
like dominoes. The Roman emperor
Marcus Aurelius seemed to anticipate particle physics when he said,
"All things are implicated with one
another, and the bond is holy; there
is hardly anything unconnected
with another thing" [6, p. 94].
Hannah Arendt, in her book
The Human Condition, reflected
on the implications of interconnection. "Action," she wrote, "no matter

september 2015

what its specific content, always
nology can escape our control.
establishes relationships and thereFor much of history, he wrote, the
fore has an inherent tendency to
ramifications of misplaced intenforce open all limitations and cut
tions were relatively limited, simply
across all boundaries."
because the scope of our reach
1) "These consequences are boundwas relatively limited. Human impoless, because action, though it
tence, he wrote, "hitherto shielded
may proceed from nowhere, so to
us from the full destructive impact
speak, acts into a medium where
of human folly." Modern technoloevery reaction becomes a chain
gies were in the process of changreaction and where every process
ing that. "The penalties for errors
is the cause of new processes.
of foresight, great as they are now,"
Since action acts upon beings
Wiener wrote, "will be enormously
who are capable of their own acincreased as automatization comes
tions, reaction, apart from being
into full use" [8].
a response, is always a
new action that strikes
out on its own and affects others. Thus action
Wiener well knew that technology
and reaction among
men never move in a
can go awry even when deployed
closed circle and can
never be reliably conby the most responsible of
fined to two partners" [6,
engineers, simply because
p. 95]. (I have expanded
on Winner's quotation
once unleashed, technology
from Arendt. See [7].
With his understandinteracts with its environment
ing that information and
in ways that are impossible to
feedback are the lingua
f ranc a of both or ga n fully predict or control.
isms and machines, Wiener recognized that the
chain reactions to which
Arendt referred apply not only to
Speed, Wiener frequently argued,
the relationships between human
is the most basic reason this is true.
beings, but also to the relationIn a 1960 essay for Science magaships bet ween huma n beings
zine, "Some Moral and Technical
and machines, and to the relaConsequences of Automation," he
t ion sh ip s be t ween mach i ne s
wrote that although we may in theoand machines. For that reason
ry be able to control our machines,
he also recognized that, thanks
such control means little if we can't
to the growing power, ubiquity,
impose it until it's too late. "By
and interconnectedness of our
the very slowness of our human
machines, the dream of mastery
actions, our effective control of our
has become one of the most danmachines may be nullified," Wiegerous illusions of our time.
ner said. "By the time we are able
In his final book, God and
to react to information conveyed
Golem, Inc., published posthuby our senses and stop the car we
mously in 1964, Wiener cited The
Sorcerer's Apprentice as a literary
object lesson in how quickly tech(continued on page 72)
∕

IEEE Technology and Society Magazine

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2015