IEEE Power & Energy Magazine - May/June 2022 - 42

Demonstration of
Intelligent HVAC
Load Management
With Deep
Reinforcement
Learning
By Yan Du, Fangxing Li, Kuldeep Kurte,
Jeffrey Munk, and Helia Zandi
B
42
BUILDINGS ACCOUNT FOR 40% OF TOTAL PRIMARY
energy consumption and 30% of all CO2 emissions worldwide.
A large portion of building energy consumption is due
to heating, ventilation, and air-conditioning (HVAC) systems.
In the summer, for example, more than 50% of a building's
electricity consumption is used for cooling. With proper
energy management, buildings can provide load shifting,
peak shaving, frequency regulation, and many other demand
response services.
Many existing approaches for building energy management
are model based and require the modeling of the complex
Digital Object Identifier 10.1109/MPE.2022.3150825
Date of current version: 19 April 2022
ieeepower & energy magazine
thermal dynamics of the HVAC system and its interaction
with the ambient environment. The development of such a
model may introduce measurement and prediction errors,
which may undermine the control performance. In addition,
models developed for one building may not generalize well for
other buildings or unseen operation environments.
In contrast to the model-based approaches, model-free
algorithms require no prior knowledge of the physical
model, such as the thermal-dynamic model in the HVAC
control case; rather, they learn the model through estimation
and exploration. One representative model-free approach is
reinforcement learning (RL). As shown in Figure 1, an agent
(e.g., a demand response controller) interacts with an environment
(for instance, the building). At each control time
1540-7977/22©2022IEEE
may/june 2022

IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022

Contents
IEEE Power & Energy Magazine - May/June 2022 - Cover1
IEEE Power & Energy Magazine - May/June 2022 - Cover2
IEEE Power & Energy Magazine - May/June 2022 - Contents
IEEE Power & Energy Magazine - May/June 2022 - 2
IEEE Power & Energy Magazine - May/June 2022 - 3
IEEE Power & Energy Magazine - May/June 2022 - 4
IEEE Power & Energy Magazine - May/June 2022 - 5
IEEE Power & Energy Magazine - May/June 2022 - 6
IEEE Power & Energy Magazine - May/June 2022 - 7
IEEE Power & Energy Magazine - May/June 2022 - 8
IEEE Power & Energy Magazine - May/June 2022 - 9
IEEE Power & Energy Magazine - May/June 2022 - 10
IEEE Power & Energy Magazine - May/June 2022 - 11
IEEE Power & Energy Magazine - May/June 2022 - 12
IEEE Power & Energy Magazine - May/June 2022 - 13
IEEE Power & Energy Magazine - May/June 2022 - 14
IEEE Power & Energy Magazine - May/June 2022 - 15
IEEE Power & Energy Magazine - May/June 2022 - 16
IEEE Power & Energy Magazine - May/June 2022 - 17
IEEE Power & Energy Magazine - May/June 2022 - 18
IEEE Power & Energy Magazine - May/June 2022 - 19
IEEE Power & Energy Magazine - May/June 2022 - 20
IEEE Power & Energy Magazine - May/June 2022 - 21
IEEE Power & Energy Magazine - May/June 2022 - 22
IEEE Power & Energy Magazine - May/June 2022 - 23
IEEE Power & Energy Magazine - May/June 2022 - 24
IEEE Power & Energy Magazine - May/June 2022 - 25
IEEE Power & Energy Magazine - May/June 2022 - 26
IEEE Power & Energy Magazine - May/June 2022 - 27
IEEE Power & Energy Magazine - May/June 2022 - 28
IEEE Power & Energy Magazine - May/June 2022 - 29
IEEE Power & Energy Magazine - May/June 2022 - 30
IEEE Power & Energy Magazine - May/June 2022 - 31
IEEE Power & Energy Magazine - May/June 2022 - 32
IEEE Power & Energy Magazine - May/June 2022 - 33
IEEE Power & Energy Magazine - May/June 2022 - 34
IEEE Power & Energy Magazine - May/June 2022 - 35
IEEE Power & Energy Magazine - May/June 2022 - 36
IEEE Power & Energy Magazine - May/June 2022 - 37
IEEE Power & Energy Magazine - May/June 2022 - 38
IEEE Power & Energy Magazine - May/June 2022 - 39
IEEE Power & Energy Magazine - May/June 2022 - 40
IEEE Power & Energy Magazine - May/June 2022 - 41
IEEE Power & Energy Magazine - May/June 2022 - 42
IEEE Power & Energy Magazine - May/June 2022 - 43
IEEE Power & Energy Magazine - May/June 2022 - 44
IEEE Power & Energy Magazine - May/June 2022 - 45
IEEE Power & Energy Magazine - May/June 2022 - 46
IEEE Power & Energy Magazine - May/June 2022 - 47
IEEE Power & Energy Magazine - May/June 2022 - 48
IEEE Power & Energy Magazine - May/June 2022 - 49
IEEE Power & Energy Magazine - May/June 2022 - 50
IEEE Power & Energy Magazine - May/June 2022 - 51
IEEE Power & Energy Magazine - May/June 2022 - 52
IEEE Power & Energy Magazine - May/June 2022 - 53
IEEE Power & Energy Magazine - May/June 2022 - 54
IEEE Power & Energy Magazine - May/June 2022 - 55
IEEE Power & Energy Magazine - May/June 2022 - 56
IEEE Power & Energy Magazine - May/June 2022 - 57
IEEE Power & Energy Magazine - May/June 2022 - 58
IEEE Power & Energy Magazine - May/June 2022 - 59
IEEE Power & Energy Magazine - May/June 2022 - 60
IEEE Power & Energy Magazine - May/June 2022 - 61
IEEE Power & Energy Magazine - May/June 2022 - 62
IEEE Power & Energy Magazine - May/June 2022 - 63
IEEE Power & Energy Magazine - May/June 2022 - 64
IEEE Power & Energy Magazine - May/June 2022 - 65
IEEE Power & Energy Magazine - May/June 2022 - 66
IEEE Power & Energy Magazine - May/June 2022 - 67
IEEE Power & Energy Magazine - May/June 2022 - 68
IEEE Power & Energy Magazine - May/June 2022 - 69
IEEE Power & Energy Magazine - May/June 2022 - 70
IEEE Power & Energy Magazine - May/June 2022 - 71
IEEE Power & Energy Magazine - May/June 2022 - 72
IEEE Power & Energy Magazine - May/June 2022 - 73
IEEE Power & Energy Magazine - May/June 2022 - 74
IEEE Power & Energy Magazine - May/June 2022 - 75
IEEE Power & Energy Magazine - May/June 2022 - 76
IEEE Power & Energy Magazine - May/June 2022 - 77
IEEE Power & Energy Magazine - May/June 2022 - 78
IEEE Power & Energy Magazine - May/June 2022 - 79
IEEE Power & Energy Magazine - May/June 2022 - 80
IEEE Power & Energy Magazine - May/June 2022 - 81
IEEE Power & Energy Magazine - May/June 2022 - 82
IEEE Power & Energy Magazine - May/June 2022 - 83
IEEE Power & Energy Magazine - May/June 2022 - 84
IEEE Power & Energy Magazine - May/June 2022 - 85
IEEE Power & Energy Magazine - May/June 2022 - 86
IEEE Power & Energy Magazine - May/June 2022 - 87
IEEE Power & Energy Magazine - May/June 2022 - 88
IEEE Power & Energy Magazine - May/June 2022 - 89
IEEE Power & Energy Magazine - May/June 2022 - 90
IEEE Power & Energy Magazine - May/June 2022 - 91
IEEE Power & Energy Magazine - May/June 2022 - 92
IEEE Power & Energy Magazine - May/June 2022 - Cover3
IEEE Power & Energy Magazine - May/June 2022 - Cover4
https://www.nxtbook.com/nxtbooks/pes/powerenergy_gridedge_2023
https://www.nxtbook.com/nxtbooks/pes/powerenergy_050622
https://www.nxtbook.com/nxtbooks/pes/powerenergy_030422
https://www.nxtbook.com/nxtbooks/pes/powerenergy_010222
https://www.nxtbook.com/nxtbooks/pes/powerenergy_111221
https://www.nxtbook.com/nxtbooks/pes/powerenergy_091021
https://www.nxtbook.com/nxtbooks/pes/powerenergy_070821
https://www.nxtbook.com/nxtbooks/pes/powerenergy_050621
https://www.nxtbook.com/nxtbooks/pes/powerenergy_030421
https://www.nxtbook.com/nxtbooks/pes/powerenergy_010221
https://www.nxtbook.com/nxtbooks/pes/powerenergy_111220
https://www.nxtbookmedia.com