IEEE Technology and Society Magazine - September 2021 - 88

Feature
* What elements constitute the discourse of sociotechnical
imaginaries being constructed about
AI within U.S. intelligence in this document?
Who and what have been involved in constructing
these imaginaries? What is still in flux or contested?
How are these imaginaries encountering
real-world issues on the ground?
* How are ethical, social, and political implications
getting built into the AI strategies that the IC
is pursuing? What are the histories shaping these
strategies and decision-making?
* What are the varied problems in intelligence that
these imaginaries are trying to solve? Are there
alternatives that are being ignored?
* How are social factors (e.g., IC cultures; intelligence
practitioner preferences; concerns such as
privacy, safety, accountability, transparency, and
trust in algorithms; organizational impacts) being
considered as part of the strategy development
and implementation of AI technologies? What
gaps and oversights exist?
Qualitative research methods used for this type of
inquiry could involve document analysis, as well as
semistructured interviews and focus groups of intelligence
practitioners and officials involved in creating
these strategy and policy documents.
In addition to looking at strategy documents,
one could also conduct case studies of specific big
data/AI projects program being developed within
the IC. For example, DARPA's Explainable Artificial
Intelligence (XAI) aims to address the central
challenge regarding the effectiveness of AI systems,
which promise to able to acquire information, learn,
make decisions, and act on their own: they are currently
unable to explain their decisions or actions to
humans [61]. As a result, there is a growing recognition
of the need for what is known as " Explainable
AI, " so that users can trust the outputs of these systems
by having a more confident understanding of
how the AI system produced its result. The DARPA
XAI program aims to create a suite of machine learning
techniques that will produce more accurate
and explainable models of AI outputs, coupled to
human-computer interfaces. These will explain the
logic behind the output and outline the strengths/
weaknesses/limitations of the output, to " enable
human users to understand, appropriately trust, and
effectively manage the emerging generation of artificially
intelligent partners " [62].
88
This XAI project would be interesting to study
because there are significant issues of trust with
regard to AI-enabled systems for intelligence analysis
relating to AI research and work being done both
inside and outside the IC. So, this would be a useful
case study in which to examine these issues as
the technology is being created. A case study of this
project could involve: 1) collection and analysis of
open-source administrative documents and publications
on XAI; 2) interviews with individuals who
are the academic AI developers, DARPA program
managers, and relevant U.S. intelligence officials,
to understand the history and rationale behind the
XAI project, a technical understanding of what XAI
aims to accomplish, and how XAI addresses current
workplace challenges that intelligence analysts face;
and 3) participant observation of the academic labs
conducting some of this research.
The participant observation part of the project in
3) could also probe the various social implications
of XAI as it might be deployed in an intelligence
work environment. How have issues of trust, privacy,
accountability, and transparency been conceptualized
as part of this project? Given that AI analyzes
data on a scale and at a speed that humans cannot
match, how much autonomy/oversight would and
should AI intelligence systems and algorithms be
given? What provisions for human oversight of and
accountability for automated analysis are being
discussed? How freely should such technology be
shared with other countries-even our allies? It is
well known that AI often encodes human biases of
race and gender, so how is the IC planning to address
that problem with these AI systems? What other
implications does XAI (and technologies similar to
it) have for the intelligence workplace? This kind of
study could explore important themes and concerns
that a diverse group of intelligence employees (analysts,
managers), in different organizational settings,
have around the deployment of AI technologies
in their workplaces (e.g., exploring issues of trust,
accountability, privacy, and transparency).
This is just one example to illustrate the types of
studies that could be pursued. Keep in mind, the
U.S. IC consists of at least 17 different agencies, each
with its own unique mission, workforce, culture, and
priorities for IT applications for their workforce. One
could imagine a plethora of micro-level case studies
(of work within particular agencies) coupled to
macro-level insights (e.g., examination of agency
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - September 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2021
