IEEE Technology and Society Magazine - June 2020 - 51

translations in the legal domain, even
if there were no legal documents in the
initial training texts [3].

of training ML models, as well as
reducing training time and therefore, energy usage.
5) Use our ML Emissions Calculator: By inputting details regarding the training of an ML model,
such as the region of the server,
the type of GPU, and the training
time, our tool gives as output the
approximate amount of CO2eq
produced. Using our tool can
give a good estimate of the order
of magnitude of emissions produced by a given ML experiment
or set of experiments.3
6) Disclose the emissions associated with published ML results:
Nowadays, few papers disclose
the specifics of their training
approach, i.e., what infrastructure they used for training and
how long it took to obtain their
results. Publishing both these
details and the overall emissions
generated by ML experiments is
important for raising awareness
around the environmental footprint of ML research.

Action Items
We do not pretend to offer specific
guidelines that anyone can follow to
reduce the carbon emissions of their
models; however, given the factors
described above, there are some
concrete actions that can be taken to
reduce the carbon footprint of ML:
1) Choose Your Cloud Providers
Wisely: all of the major cloud providers have information regarding their sustainability efforts on
their websites and there are thirdparty resources that endeavor to
compare their environmental
footprints.
2) Select Data Center Location:
When requesting a cloud GPU or
CPU, it is very easy to request a
server in a specific location (e.g.,
U.S.-East, Europe-West, etc.) in
order to choose the least carbon-intensive location. There is
freely available emissions factor
documentation that provides the
carbon emitted by energy grids
worldwide.
3) Reduce Wasted Resources:
Carrying out a literature review
before starting experimentation,
using pretrained mod--els when
possible, and using random
search instead of grid search can
reduce the quantity of failed experiments needed to obtain the best
results, and therefore the footprint
of the model as a whole.
4) Choose More Efficient Hardware: Recent generations of
computing hardware such as
GPUs and tensor processing
units (TPUs) have been specifically designed for the parallel
computations involved in training neural networks. Using this
hardware instead of traditional
chips can improve the efficiency
JUNE 2020

Insight into Environmental
Impact of ML
We have discussed some major factors and considerations in the current article, enabling Machine Learning practitioners to have some insight
regarding the environmental impact
of training of their models. We realize
that it is not always possible to take
all of these factors into consideration
during ML practice, which brings with
it constraints such as privacy and
data accessibility, but these considerations are useful to keep in mind as
a guiding thread towards more sustainable ML research and practice.
3

While we endeavor to estimate the quantity
of CO2 produced by the energy usage involved
in training an ML model, this is a simplified
estimation of the total CO2 produced by a given
ML model, which would also need to include an
extensive Life Cycle Assessment of the hardware used during the process, as well as the
emissions produced during inference time.

∕

IEEE TECHNOLOGY AND SOCIETY MAGAZINE

Our hope is that estimating and disclosing the quantity of carbon emissions produced by ML models will
increasingly become a more mainstream phenomenon and part of the
ML research process, similar to the
way in which sharing code and data
has increasingly become the norm in
recent years. In order to facilitate this
even further, we are currently working to create an easy-to-install python
package that will allow seamless
tracking during experimentation
time. We believe that our work, along
with that of others, will open the
door to measuring the environmental impact of our field, and for making positive changes in order to
reduce those impacts.

Author Information
Alexandra Luccioni is with Mila, and
with the Université de Montréal,
Montreal, Quebec, Canada.
Alexandre Lacoste is with Element AI, Montreal, Canada.
Victor Schmidt is with MILA, Montreal, Canada.

References
[1] D. Guyon, A.C. Orgerie, C. Morin, and D.
Agarwal, "How much energy can green HPC
cloud users save?," in Proc. 2017 25th Euromicro Int. Conf. Parallel, Distributed, and
Network-Based Processing (PDP). IEEE, Mar.
2017, pp. 416-420.
[2] N. Hill, C. Dun, R. Watson, and K. James,
"2015 Government GHG conversion factors for
company reporting: Methodology paper for emission factors," final rep., Department of Energy and Climate Change (DECC), London,
U.K., 2015.
[3] J. Howard and S. Ruder, "Universal language
model fine-tuning for text classification," arXiv
preprint arXiv:1801.06146, 2018.
[4] J. Koomey, "Growth in data center electricity
use 2005 to 2010," Analytical Press, report
completed at the request of The New York
Times, vol. 9, p. 161, 2011.
[5] I. Kemelmacher-Shlizerman, S.M. Seitz, D.
Miller, and E. Brossard, "The megaface benchmark: 1 million faces for recognition at scale," in
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 4873-4882.
[6] E. Strubell, A. Ganesh, and A. McCallum, "Energy and policy considerations
for deep learning in NLP," arXiv preprint
arXiv:1906.02243, 2019.

IEEE Technology and Society Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2020