IEEE Power & Energy Magazine - May/June 2022 - 49

temperature violation (i.e., between
5,000 and 6,000 min in the figure),
mainly due to the low setpoint. In
contrast, the DDPG-based control
strategy does not place the setpoint
at the lowest possible value even
during peak price hours to avoid
comfort violations during the time
of extremely low outdoor temperatures.
These comparisons show that,
once well trained, the DDPG control
has learned the impacts of the price
signal and outdoor temperature on
the reward, and it develops an intelligent
setpoint control strategy to
accommodate both the price peak
and low outdoor temperature.
Figure 6(c) shows the fixed-setpoint
case, so-called because this
control strategy always sets the
temperature at the highest value.
Thus, the indoor temperature also
remains at the highest level among
the three control strategies. Consequently,
this fixed-setpoint control
leads to the highest energy cost.
Note that, when calculating temperature
or comfort violations, only
the time when the indoor temperature
is below the lower bound is
counted. The reason is that this
is a heating scenario, and a low
indoor temperature is considered an
unbearable violation, while a high
indoor temperature is acceptable to
residential HVAC users.
In the second scenario, the
pretrained DDPG control strategy
is validated with 10 unseen building
models with different thermal
mass parameters to demonstrate
its generalization ability. Table 3
shows a comparison of the energy
costs and temperature violations
for
the DDPG control and two
aforementioned benchmark controls.
As the table shows, like in
scenario 1, the rule-based control
gives the lowest energy cost,
while the fixed-setpoint control
gives the fewest violations.
The pretrained DDPG control
achieves a balanced HVAC control
strategy that results in a relatively
lower energy cost and fewer
may/june 2022
-5
-10
1,000 2,000 3,000 4,000
Time (min)
(c)
Tin,1
Tout
Setpoint1
Price
5,000 6,000 7,000
25
20
15
10
5
-5
-10
1,000 2,000 3,000 4,000
Time (min)
(a)
25
20
15
10
5
-5
-10
1,000 2,000 3,000 4,000
Time (min)
(b)
25
20
15
10
5
5,000 6,000 7,000
5,000 6,000 7,000
ac Status: Off
ac Status: On
Tlower
Peak Price: US$0.25/kWh
Off-Peak Price: US$0.05/kWh
Temperature Violation
ac Status
figure 6. A comparison of the setpoint control strategies for the first five test days
(zone 1): (a) DDPG, (b) rule based, and (c) fixed setpoint.
ieee power & energy magazine
49
Temperature (°C)
Temperature (°C)
Temperature (°C)

IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022

Contents
IEEE Power & Energy Magazine - May/June 2022 - Cover1
IEEE Power & Energy Magazine - May/June 2022 - Cover2
IEEE Power & Energy Magazine - May/June 2022 - Contents
IEEE Power & Energy Magazine - May/June 2022 - 2
IEEE Power & Energy Magazine - May/June 2022 - 3
IEEE Power & Energy Magazine - May/June 2022 - 4
IEEE Power & Energy Magazine - May/June 2022 - 5
IEEE Power & Energy Magazine - May/June 2022 - 6
IEEE Power & Energy Magazine - May/June 2022 - 7
IEEE Power & Energy Magazine - May/June 2022 - 8
IEEE Power & Energy Magazine - May/June 2022 - 9
IEEE Power & Energy Magazine - May/June 2022 - 10
IEEE Power & Energy Magazine - May/June 2022 - 11
IEEE Power & Energy Magazine - May/June 2022 - 12
IEEE Power & Energy Magazine - May/June 2022 - 13
IEEE Power & Energy Magazine - May/June 2022 - 14
IEEE Power & Energy Magazine - May/June 2022 - 15
IEEE Power & Energy Magazine - May/June 2022 - 16
IEEE Power & Energy Magazine - May/June 2022 - 17
IEEE Power & Energy Magazine - May/June 2022 - 18
IEEE Power & Energy Magazine - May/June 2022 - 19
IEEE Power & Energy Magazine - May/June 2022 - 20
IEEE Power & Energy Magazine - May/June 2022 - 21
IEEE Power & Energy Magazine - May/June 2022 - 22
IEEE Power & Energy Magazine - May/June 2022 - 23
IEEE Power & Energy Magazine - May/June 2022 - 24
IEEE Power & Energy Magazine - May/June 2022 - 25
IEEE Power & Energy Magazine - May/June 2022 - 26
IEEE Power & Energy Magazine - May/June 2022 - 27
IEEE Power & Energy Magazine - May/June 2022 - 28
IEEE Power & Energy Magazine - May/June 2022 - 29
IEEE Power & Energy Magazine - May/June 2022 - 30
IEEE Power & Energy Magazine - May/June 2022 - 31
IEEE Power & Energy Magazine - May/June 2022 - 32
IEEE Power & Energy Magazine - May/June 2022 - 33
IEEE Power & Energy Magazine - May/June 2022 - 34
IEEE Power & Energy Magazine - May/June 2022 - 35
IEEE Power & Energy Magazine - May/June 2022 - 36
IEEE Power & Energy Magazine - May/June 2022 - 37
IEEE Power & Energy Magazine - May/June 2022 - 38
IEEE Power & Energy Magazine - May/June 2022 - 39
IEEE Power & Energy Magazine - May/June 2022 - 40
IEEE Power & Energy Magazine - May/June 2022 - 41
IEEE Power & Energy Magazine - May/June 2022 - 42
IEEE Power & Energy Magazine - May/June 2022 - 43
IEEE Power & Energy Magazine - May/June 2022 - 44
IEEE Power & Energy Magazine - May/June 2022 - 45
IEEE Power & Energy Magazine - May/June 2022 - 46
IEEE Power & Energy Magazine - May/June 2022 - 47
IEEE Power & Energy Magazine - May/June 2022 - 48
IEEE Power & Energy Magazine - May/June 2022 - 49
IEEE Power & Energy Magazine - May/June 2022 - 50
IEEE Power & Energy Magazine - May/June 2022 - 51
IEEE Power & Energy Magazine - May/June 2022 - 52
IEEE Power & Energy Magazine - May/June 2022 - 53
IEEE Power & Energy Magazine - May/June 2022 - 54
IEEE Power & Energy Magazine - May/June 2022 - 55
IEEE Power & Energy Magazine - May/June 2022 - 56
IEEE Power & Energy Magazine - May/June 2022 - 57
IEEE Power & Energy Magazine - May/June 2022 - 58
IEEE Power & Energy Magazine - May/June 2022 - 59
IEEE Power & Energy Magazine - May/June 2022 - 60
IEEE Power & Energy Magazine - May/June 2022 - 61
IEEE Power & Energy Magazine - May/June 2022 - 62
IEEE Power & Energy Magazine - May/June 2022 - 63
IEEE Power & Energy Magazine - May/June 2022 - 64
IEEE Power & Energy Magazine - May/June 2022 - 65
IEEE Power & Energy Magazine - May/June 2022 - 66
IEEE Power & Energy Magazine - May/June 2022 - 67
IEEE Power & Energy Magazine - May/June 2022 - 68
IEEE Power & Energy Magazine - May/June 2022 - 69
IEEE Power & Energy Magazine - May/June 2022 - 70
IEEE Power & Energy Magazine - May/June 2022 - 71
IEEE Power & Energy Magazine - May/June 2022 - 72
IEEE Power & Energy Magazine - May/June 2022 - 73
IEEE Power & Energy Magazine - May/June 2022 - 74
IEEE Power & Energy Magazine - May/June 2022 - 75
IEEE Power & Energy Magazine - May/June 2022 - 76
IEEE Power & Energy Magazine - May/June 2022 - 77
IEEE Power & Energy Magazine - May/June 2022 - 78
IEEE Power & Energy Magazine - May/June 2022 - 79
IEEE Power & Energy Magazine - May/June 2022 - 80
IEEE Power & Energy Magazine - May/June 2022 - 81
IEEE Power & Energy Magazine - May/June 2022 - 82
IEEE Power & Energy Magazine - May/June 2022 - 83
IEEE Power & Energy Magazine - May/June 2022 - 84
IEEE Power & Energy Magazine - May/June 2022 - 85
IEEE Power & Energy Magazine - May/June 2022 - 86
IEEE Power & Energy Magazine - May/June 2022 - 87
IEEE Power & Energy Magazine - May/June 2022 - 88
IEEE Power & Energy Magazine - May/June 2022 - 89
IEEE Power & Energy Magazine - May/June 2022 - 90
IEEE Power & Energy Magazine - May/June 2022 - 91
IEEE Power & Energy Magazine - May/June 2022 - 92
IEEE Power & Energy Magazine - May/June 2022 - Cover3
IEEE Power & Energy Magazine - May/June 2022 - Cover4
https://www.nxtbook.com/nxtbooks/pes/powerenergy_gridedge_2023
https://www.nxtbook.com/nxtbooks/pes/powerenergy_050622
https://www.nxtbook.com/nxtbooks/pes/powerenergy_030422
https://www.nxtbook.com/nxtbooks/pes/powerenergy_010222
https://www.nxtbook.com/nxtbooks/pes/powerenergy_111221
https://www.nxtbook.com/nxtbooks/pes/powerenergy_091021
https://www.nxtbook.com/nxtbooks/pes/powerenergy_070821
https://www.nxtbook.com/nxtbooks/pes/powerenergy_050621
https://www.nxtbook.com/nxtbooks/pes/powerenergy_030421
https://www.nxtbook.com/nxtbooks/pes/powerenergy_010221
https://www.nxtbook.com/nxtbooks/pes/powerenergy_111220
https://www.nxtbookmedia.com