IEEE Technology and Society Magazine - March 2020 - 21

subsequent model behav ior by
measuring fairness metrics such as
Equal Opportunity and Demographic Parity [47]. Other approaches address bias by changing the training
procedure or the structure of the ML
models, for instance by transforming the raw data in a space in which
discriminatory information cannot
be found [49], or using a variational autoencoder to learn the latent
structure from the dataset and using
this structure to re-weight the importance of specific data points during
model training [33]. Whatever the approach chosen, using these kinds of
tools during ML model development
and deployment can change the life
of individual people, who could go
from unfairly spending decades in
prison to having the chance of a better life - an immensely important
difference especially when multiplied by the thousands of people
whose lives can be affected by the
deployment of these tools. This
multiplication of bias is especially
important to consider since ML is
being used more and more, and
therefore even edge cases and small
minorities can be amplified in realworld applications.

of algorithmic fairness, especially
if algorithms such as this one make
filtering or hiring decisions that can
ultimately affect an entire gender's
lives and careers. This can potentially create a negative feedback loop,
as such a system would reduce the
number of female workers and thus
the number of positive role models
for girls interested in technology. A
similar type of gender bias was also
found in pretrained word embedding models, which were found to
exhibit gender stereotypes in terms
of higher cosine similarity between,
for instance, "woman" and "homemaker" or "receptionist" as opposed
to "woman" and "doctor" or "lawyer,"
notably due to these biases existing
in the corpus that they were trained
on, which consisted of mainstream
news articles [4].
In order to reduce and eventually remove gender bias in written
text, researchers have proposed approaches such as identifying the
gender subspace of vectors and
adjusting the dimensions in a way
that either neutralizes or entirely removes gender bias [4]. Others have
defined a formal gender bias taxonomy in order to capture gender
bias and to train ML models to later
identify this bias in texts [18]. Debiasing the computational representation
of language, notably word embedding models, is especially important
because of the extent of their usage; pretrained embedding models
trained on corpora such as Google
News and the Common Crawl are
used in a variety of applications and
systems, and can therefore continue
perpetuating gender bias in downstream usages in Natural Language
Processing (NLP) applications such as
dialogue systems. This is a challenge
given the complex and subsymbolic
nature of modern NLP, which makes
it difficult to analyze specific features
and aspects of data and identify latent connections and bias between

Textual Bias
Bias is not always in numbers, it can
also manifest itself in the words that
we use to describe the world around
us. For instance, in 2018, Reuters
reported that Amazon was forced
to decommission an ML-powered
recruiting engine when it was discovered that it penalized any mention of female-related vocabulary,
including applicants who attended
all-women colleges [9]. This is not
surprising given the gender disparity that exists in the technology
sector and since the data used to
develop this tool was comprised of
resumes submitted (and accepted)
to Amazon over a 10-year period. It
is nonetheless disturbing in terms
MARCH 2020

∕

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

words and concepts. Therefore, more
work is needed to explore and analyze these issues, which constitutes
an interesting research direction in itself, and one that is important to pursue and to integrate into mainstream
ML research.
Despite the research initiatives
described above to carve appropriate social norms about AI, there
remains a noticeable gap between
the recommendations they make
and ways to ensure that these are
respected. Legislation of AI is still
catching up to the progress made
in research and practice, and there
have not yet been any countr ylevel laws governing AI research
specifically. However, there have
been, on the one hand, more highlevel legislative frameworks such as
the European Union (EU) General
Data Protection Regulation (GDPR)
(https://gdpr-info.eu/), which aims to
ensure data privacy and protection
and, on the other hand, more local
initiatives such as San Francisco's
Facial Recognition Software Ban.
Nonetheless, more complete legal
frameworks are needed to control
nefarious use of AI and to ensure
that the principles defined in theory
are applied and enforced in practice.

AI for Good Initiatives
Whereas the profit motive is the main
driver behind much of the commercial deployment of AI today, there are
nonetheless many projects going on
in academia, government organizations, civil society, and industry labs
motivated by more noble objectives,
often called AI for Social Good (AISG)
projects. In addition to the specific
projects being undertaken in areas
such as healthcare, education, or
the environment, it is interesting to
highlight higher-level efforts that
aim to foster and facilitate these
projects. For example, the AI Commons project (https://ai-commons
.org/) aims to construct a hub where

21
https://www.gdpr-info.eu/ https://ai-commons.org https://ai-commons.org
IEEE Technology and Society Magazine - March 2020

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2020
