IEEE Technology and Society Magazine - March 2021 - 73

Examples
In the sections below, we illustrate examples of
applications in which biases result (or might result)
in increased inequality.

Diagnosis
There are many algorithmic approaches for supporting COVID-19 diagnosis from computer tomography (CT) and X-ray scans which frame the diagnostic
problem as a classification task (i.e., identifying
healthy vs. COVID-19-positive individuals), to training neural networks to detect masses and patterns in
lung scans [17]-[19]. This can be particularly problematic when attempting to make meaningful conclusions based solely on medical imagery and, in
particular, in geographies with a high prevalence of
other diseases not included in the training data sets
which might also affect the lungs (such as tuberculosis or HIV/AIDS [20]-[22]), which can be confused
with COVID-19 and lead to its misdiagnosis. In addition, much of the existing medical imaging research
relies on small and poorly balanced data sets that
mix data from several populations without proper
traceability. Prior research has shown that when
training data sets are imbalanced on gender, the
performance of deep learning models in radiology
decreases, in particular, in the case of X-ray image
data sets used to diagnose thoracic diseases [23].
Such biases might also reduce the performance of
AI applications on CT and X-ray imaging of COVID-19
if not properly taken into account.
Another structural source of bias is that not all
regions can afford scanner equipment. As a result,
the data used to train the related AI models will not
be representative and the approaches proposed may
not be systematically deployed in disadvantaged
regions. On the other end of the technology requirement spectrum, there are some proposals involving
mobile-based diagnosis approaches [24] that are
potentially promising, but warrant further exploration and validation.
Much of the existing work that utilizes AI to analyze medical imagery does not provide transparency
or interpretability, delivering a categorical verdict
based solely on an incoming image. This " blackbox " approach may be acceptable for human-in-theloop deployment where potential COVID-19-positive
images are flagged for expert radiologists, who will
then carry out further analysis manually. However,
such approaches can be problematic in contexts

March 2021

where medical experts are lacking or do not have
enough time. Fully automated AI pipelines must be
assessed in the context of their clinical impact before
being deployed, which includes carefully considering and mitigating the risks in addition to assessing
biases which might result into incorrect and unfair
functioning.
Utilizing complementary medical data such
as information regarding a patient's gender, age,
and comorbidities, as well as clinical indicators,
does not only improve the accuracy of image-only
approaches, but also produces results that are more
interpretable for clinicians. For instance, a hybrid
approach that merges algorithmic analysis of both
CT scans as well as clinical features to predict the
severity of COVID-19 [25] reported high accuracy
rates, and the set of clinical features identified as
relevant by the algorithm were coherent with those
identified by previous studies. This overlap is promising for eventual clinical monitoring of COVID-19
severity, both manually and using Al-infused
approaches. However, these kinds of hybrid studies
should also be replicated in other regions and complemented with clinical data from incoming cases
around the world, achieving better global coverage
and reproducibility.

Treatment
Research has consistently shown higher rates
of infection, hospitalization, and death in ethnic
minorities during the COVID-19 pandemic [12],
[13], [26], [27]. Nonetheless, our understanding of
these inequalities remains poor, which hinders the
development of solutions to combat COVID-19 and
the disparities it magnifies. Gathering sufficient and
accurate data on all of the social determinants of
health, including race and ethnicity, is critical for
effective research and development of both medical and public health interventions. However, inadequate and biased data collection is prevalent in
practice. For example, data collection mechanisms
are often poorly designed, with inconsistent ethnicity and race labeling [28], [29]. In fact, a systematic
review has found that of 1518 COVID-19-related clinical trials registered on ClinicalTrials.gov, only one
randomized controlled trial and five observational
studies collected data on ethnicity [30]. Overall,
fragmented and incomplete data makes it challenging for AI to succeed in furthering our understanding
of COVID-19 and devising appropriate interventions.

http://www.ClinicalTrials.gov

IEEE Technology and Society Magazine - March 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - March 2021