IEEE Technology and Society Magazine - June 2021 - 84

Feature
The disciplinary divide
Another challenge related to AI's complexity is the
plurality of professional disciplines with roles to play
in shaping responsible AI, beyond engineers and
computer scientists. Indeed, discourse on responsible
AI has been advanced by ethicists, historians
and philosophers of technology, journalists, policy
analysts, political decision-makers, other social scientists,
members of the public, and more. Yet these
diverse stakeholders may differ in their technical
and ethical education, their framing of problems
and solutions, their attitudes and values toward
responsible AI, and their norms of communication.
Consider attempts to apply the principle of fairness
in attempting to minimize bias. Arguably,
a thoughtful AI engineer today might identify a
normative principle like " fairness, " specified in a
corporate responsible AI policy, pick a plausible
fairness metric to instantiate it (noting there are
ineliminable tradeoffs between different metrics
[27]), apply it, and communicate these decisions
transparently [28].
Yet approaching social issues like bias and fairness
too narrowly leads to what Selbst et al. (2018)
call category or abstraction errors [29]. Computer
scientists developing AI systems may fail to consider
how an AI system will be implemented in different
social contexts, influence human behavior in those
contexts, or lead to long-term ripple effects, all of
which can threaten the assumptions on which the AI
system is built. This is especially difficult as predicting
a technology's usage and impact is known by historians
of science and technology to be difficult [30].
Consider an algorithm used to inform a judge's
decision about criminal sentencing. An algorithm
designed and trained on test data from one jurisdiction
may translate poorly to another. It may influence
the judge's decisions in unexpected ways that
are not often accounted for [31], as a judge may
overtrust or undertrust the algorithm, or even hold
values contrary to those reflected in the algorithm.
The consequences for criminal justice outcomes
when such a system is used in complex contexts is
unclear, and may feed back in unexpected or problematic
ways if an AI is trained on data the systems
has itself helped to generate. To reiterate, there
are many questions about responsible AI that cannot
be straightforwardly addressed with a narrow
technical lens.
84
On the other hand, social scientists may bring a
lens that faces an inverse problem to that faced by
engineers: frameworks for considering social and
ethical consequences of AI more in line with the
thinking of social scientists can be sufficiently broad
but also unhelpfully complex and vague, making
them difficult to translate into practices. For example,
ethicists recognize that concepts like justice are
complex, while political scientists know that values
surrounding justice are politically contested [32].
Yet AI engineers must define some measure of justice
to implement it.
In addressing issues like inequality, social scientists
may propose large structural changes to economic
and social systems, some of which are difficult
to achieve (e.g., reforming the motives of corporations)
and or even feel far-fetched (e.g., changing
the structure of capitalism), changes which may be
significantly outside of the scope of control of AI engineers.
Conceptions of AI based on sweeping, overly
futuristic, or unrealistic generalizations may also be
unhelpful. In the best case, it is difficult to resolve the
awkwardness of attempting to apply purely technical
or social fixes to fundamentally sociotechnical problems.
Something is lost in translation.
The many hands problem
It is clear that responsibly designing and applying
AI is therefore both a technical challenge and a social
one (implicating social, economic, and policy questions).
For example, creating a facial recognition system
for policing that minimizes racial bias (by some
technical measure) is inseparable from questions on
the legitimacy of the use of that system in a particular
social and policy setting. However, the question
of distributing accountability for addressing these
multi-faceted choices remains open and contested.
Engineers and computer scientists may see their
responsibility as focused on the quality and safety
of a particular product rather than on larger -scale
social issues and maybe unaware of this wider universe
of implications [33]. Business managers and
companies may view themselves as having a primary
duty to shareholders which involves emphasizing
revenue generation above broader societal impacts.
This potentially creates holes in responsibility for
addressing the key impacts of AI.
In addition to uncertainty regarding one's scope
of professional accountability, engineers and computer
scientists who focus on the design of systems
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - June 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2021
