IEEE Computational Intelligence Magazine - May 2023 - 62

FIGURE 2. Overview of sustainable AI.
the research efforts ofGreen AI also reveal
the critical need to take environmental
impacts into major consideration when
developingAI.
This section intends to address environmental
sustainability impacts on AI
by reviewing related AI techniques,
gaining deep insights into their function,
and analyzing their potential contributions
and limitations to environmental
sustainability. Due to the limited space
for the manuscript, only major and
emerging AI algorithms are covered in
the two core categories, namely, computation-efficient
AI and data-efficient AI,
which can address the sustainability challenges
ofsignificant carbon emissions for
model training and inference, and huge
efforts for data collection and annotation.
Note that both basic and advanced
papers for a specific AI technique are
searched and reviewed, but excluding
papers that violate the initial objective
(e.g., reducing carbon emissions).
A. Computation-EfficientAI
AI models have become more and
more powerful along with the dramatic
increase in their model sizes. According to
Fig. 4, the computation requirements of
AI models will double every 3.4 months2,
which is much faster than Moore'slaw
claiming to double every two years. The
substantial size of AI models has a noteworthy
impact on environmental sustainability,
as more powerful platforms are
2https://openai.com/blog/ai-and-compute/
FIGURE 3. CO2 carbon footprint benchmarks. Source data from [11], [18], [19].
62 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2023
required for their deployment and more
energy will be consumed for real-time
implementation.
To reduce the environmental
impact of AI, an effective way is to
develop computation-efficient AI that
requires less computational resources
and at the same time generates smaller
yet accurate models [22]. This has generally
been established as compressed AI
models. The compressed models could
significantly reduce storage memory
and energy for deployment, hence
directly leading to lower carbon footprint.
Many advanced techniques have
been developed to build compressed AI
models which can be divided into three
categories, i.e., pruning, quantization,
and knowledge distillation. The details
ofeach category are discussed in the following
subsections.
1) Pruning
With the consensus that AI models are
typically over-parameterized [23], [24],
[25], network pruning serves to remove
redundant weights or neurons that have
the least impact on accuracy, resulting
in significant reduction ofstorage memory
and computational cost. It is one of
the most effective methods to compress
AI models. Fig. 5 shows the process of
pruning for a typical neural network
based AI model.
Pruning criteria are essential for all
pruning approaches. One ofthe simplest
heuristics is to estimate importance in
terms of the absolute value of a weight,
called magnitude-based pruning. Zerovalued
weights or weights within certain
threshold are removed in [26], [27].
Moreover, to make the weights zero,
regularization term is often adopted
https://openai.com/blog/ai-and-compute/
IEEE Computational Intelligence Magazine - May 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2023
