IEEE Technology and Society Magazine - June 2019 - 50

data-driven policies and actions, drawing upon practical
examples from data-driven policing.
Predictive policing, as the application of AI techniques to policing data, has its roots in much older practices of collecting crime data. Yet it also has the
potential to draw upon data from other sources in
increasingly networked police departments, and
increasingly digitally surveilled communities. Ethical
questions arise at almost every stage of data collection
and analysis, from where data is collected and sensors
are placed, to how data is encoded, to existing biases in
segregated communities and policing practices, to the
ways data is used in police management and police
encounters with the public. For building a more general
approach to AI ethics, it is useful to separate these
problems out and identify the key ethical issues, and
how AI researchers and system designers might think
about and address them.

Data: From CompStat to Critical Data Science
Information and communication technologies (ICT)
have long been central to policing. From the keeping of
criminal records and crime statistics and their collection in databases, to the use of police boxes, telephones, radio dispatching, and 9-1-1 emergency call
centers, many ICT technologies have become as closely
associated with policing as badges and handcuffs. Initially, these technologies were analog-paper records,
photographs and inked fingerprints, dedicated police
telephone boxes, and wireless radios. With the computerization of businesses and government agencies from
the 1960s to 1990s, many aspects of police work also
became digitized and computerized. Police patrol cars
began getting computers in the early 1980s, which
allowed officers to check vehicle license plates, and
eventually check individuals for outstanding warrants.
The transition from paper to digital records for crime
reports soon led to interest in compiling crime statistics
at a local level for use in guiding the management of
patrols and policing priorities. CompStat, short for Comparative Statistics, was the result. Initially adopted by
the New York City police department in 1995, similar
practices have been adopted across the country, especially in large urban departments.
CompStat as a mere data gathering and management practice has not been without its critics. In 2010,
John Eterno and Eli Silverman, retired New York police
captains turned university professor and criminology
professor, respectively, published a book-length criticism of CompStat practices in the NYPD [54]. The book
argues that there was widespread misreporting of
crimes across NYPD precincts, which took the form of
downgrading the seriousness of reported crimes in an
effort to show annual improvements in serious crime

statistics. They argued that this systematic downgrading
of crime statistics was the result of pressure from police
leadership and administration. They further argued that
pressures to increase police stops, especially in the era
of "stop and frisk" in New York City, was highly racially
discriminatory. The book caused enough controversy
and embarrassment for the NYPD that the Police Commissioner ordered an independent study to review
CompStat [55]. That review did indeed find serious systemic reporting errors. It did not, however, find evidence
that this was the result of administrative pressure,
though the review did not investigate that point exhaustively, nor did it seriously assess systemic racism within
CompStat's data collection practices.
What emerges from the investigations and reports
into CompStat, from a data science and AI ethics perspective, is the susceptibility of data to political and
bureaucratic pressure. While it may be convenient to
assume that a given dataset offers an accurate representation of the world, this should not be taken for granted.
In this case there were widespread and systematic errors
in the reported data. If that data were to be used by predictive policing algorithms, those errors could have a significant impact on policing practices. And if that data is
indeed racially biased, as it most likely is, it could further
bias policing practices. But without an awareness of
these issues, and the potential for inaccurate data or
latent bias within data, the designers of those AI algorithms may be creating garbage-in-garbage-out systems,
believing that they are producing quality systems (as
measured by their available data). The lesson for AI ethics is to never take for granted the accuracy of given
data, but to be suspicious, to seek out likely ways in
which political, economic, or social pressures may have
influenced historical datasets, to consider how it may be
shaping current data collection practices, and to be sensitive to the ways in which new data practices may transform social practices and how that relates to the
communities and individuals a system aims to care for.
With the growing popularity of AI, and increasing
concerns about its impact on society, universities and
professional organizations have recognized the problem
and taken up the challenge of teaching ethics to the
next generation of AI designers. Today, many undergraduate and graduate programs teaching AI include ethical
training, but its adoption has been uneven and more
could be done. Many online and professional training
programs still lack critical design and ethical thinking in
favor of teaching the latest techniques and tools over
good design. Professional organizations including IEEE,
ACM, and AAAI have also led initiatives to develop ethical standards, codes of ethics, and organize a growing
number of conferences and workshops on AI ethics.
These are all positive developments, and it is hoped that

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

∕

JUNE 2019

IEEE Technology and Society Magazine - June 2019

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2019