IEEE Technology and Society Magazine - March 2021 - 76

Opinion

AI researchers and practitioners should pay
special attention to bias introduced in the problem definition and data collection stages. Public
health algorithms play important roles in decision-making, from shaping critical care protocols
to deciding how to distribute vaccines. Women
and minorities are also often not properly represented in data sets whose use may result in
medical treatments and services. In addition,
nonpharmaceutical interventions are increasingly
based on measures constructed from digital traces
(e.g., mobility patterns recorded from cell phone
usage), although these do not account for those
individuals without access to mobile networks or
the internet. Finally, the spread of mis- and disinformation can be amplified by AI, which impacts
more those with less health literacy.
In all of these examples, AI-driven biases and
their implications for inequality should be assessed
from the initial project development stages through
the presentation and use of outputs. Furthermore, AI
systems should be adapted to local contexts, following and accounting for cultural and social norms. It
is also important to recognize that in some cases, the
proposed solutions might be fair and efficient with
respect to a particular problem statement, but may
increase inequality along other dimensions. Contrary to other AI applications where the goal is to
identify a single model to beat a single metric and
benchmark, in situations where vulnerable populations are at risk, a more sensitive approach should
be taken with a focus on outliers as metrics to assess
failure as well as success. This analysis may require
the problem to be approached from different angles.
For example, outliers in a patient group might represent diversity and, rather than focusing on average
performance, the more relevant task for modelers might be to understand for whom the model is
performing worst.
Some key concepts to help reduce bias in AI
applications include: the transferability of models
and their adaptation to local contexts; federated
learning strategies which may make it possible
to include patient data from multiple cohorts in
a privacy-preserving way; and the deployment of
interpretable models which allow users to interrogate the patterns and reasons for model decisions, which is particularly relevant in high-risk
settings. We hope this article will stimulate some

reflections in the AI community aiming to support the COVID-19 response and will encourage
researchers to assess whether biases in their AI
applications will amplify inequality from a health,
economic, and social perspective.


Acknowledgments
The United Nations Global Pulse is supported
in part by the Governments of Sweden and Germany and in part by the William and Flora Hewlett
Foundation. The work of Joseph Bullock was supported by the United Kingdom Research and Innovation - Science and Technology Facilities Council
(UKRI-STFC) under Grant ST/P006744/1. The work of
Alexandra Luccioni was supported by funding from
IVADO and Mila.

 References
[1] L. Farré et al., " How the COVID-19 lockdown affected
gender inequality in paid and unpaid work in Spain, "
IZA Inst. Labor Econ., Bonn, Germany, Tech. Rep. IZA
DP no. 13434, 2020.
[2] C. Rivera et al., Gender Inequality and the COVID-19
Crisis: A Human Development Perspective. New
York, NY, USA: United Nations Development
Programme, 2020.
[3] T. Power et al., " COVID-19 and indigenous peoples:
An imperative for action, " J. Clin. Nursing, vol. 29,
nos. 15-16, pp. 2737-2741, Aug. 2020, doi: 10.1111/
jocn.15320.
[4] N. A. Ashford et al., " Addressing inequality: The first
step beyond COVID-19 and towards sustainability, "
Sustainability, vol. 12, no. 13, p. 5404, Jul. 2020.
[5] UN. (2020). A UN Framework for the Immediate SocioEconomic Response to COVID-19. [Online]. Available:
https://unsdg.un.org/sites/default/files/2020-04/
UN-framework-for-the-immediate-socio-economicresponse-to-COVID-19.pdf
[6] J. Bullock et al., " Mapping the landscape of artificial
intelligence applications against COVID-19, " J. Artif.
Intell. Res., vol. 69, pp. 807-845, 2020.
[7] WHO. (2020). Draft Landscape of COVID-19
Candidate Vaccines. [Online]. Available: https://www.
who.int/who-documents-detail/draft-landscape-ofcovid-19-candidate-vaccines
[8] M. Weinstock, A. Echenique, and J. E. A. Russell,
" Chest X-ray findings in 636 ambulatory patients with
COVID-19 presenting to an urgent care center: A
normal chest X-ray is no guarantee, " J. Urgent Care
Med., vol. 14, no. 7, pp. 13-18, 2020.

IEEE Technology and Society Magazine

https://unsdg.un.org/sites/default/files/2020-04/UN-framework-for-the-immediate-socio-economic-response-to-COVID-19.pdf https://unsdg.un.org/sites/default/files/2020-04/UN-framework-for-the-immediate-socio-economic-response-to-COVID-19.pdf https://unsdg.un.org/sites/default/files/2020-04/UN-framework-for-the-immediate-socio-economic-response-to-COVID-19.pdf https://www.who.int/who-documents-detail/draft-landscape-of-covid-19-candidate-vaccines https://www.who.int/who-documents-detail/draft-landscape-of-covid-19-candidate-vaccines https://www.who.int/who-documents-detail/draft-landscape-of-covid-19-candidate-vaccines

IEEE Technology and Society Magazine - March 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2021