IEEE Power & Energy Magazine - May/June 2022 - 52

With these many different conditions affecting the achievable
power response, the system is difficult to accurately model, making
it a challenging real-world application for the deep RL approach.
✔ Next, we re-create the DQN's deployment (in simulation)
by running the DQN's setpoints through this
building simulation for the duration of 1 March-5
April 2021. This is referred to as the " simulated DQN "
case, stemming from the actual DQN deployment.
✔ Then, we use this fine-tuned building simulation for
the baseline case by using a fixed setpoint of 22.2 °C
for the same duration. This later run mimics the deployment
of the fixed-setpoint baseline control in the
research house during the deployment period.
It should be noted that, since we do not have any measured
data for the baseline case to use for setting the initial
temperatures in the simulation, we instead simulate an additional
3.5 weeks (i.e., 1-24 March 2021) of baseline operation
before the period of interest. Simulating this additional
time allows the modeled temperatures to stabilize to realistic
values and minimizes the effect of any error associated with
the selected initial temperatures.
The results from the " simulated DQN " case using the
retrained model and baseline simulation case employing
the fixed-setpoint control are shown in Figure 8(a)
and (b), respectively. In the top plot of Figure 8(a), the
Tin1_sim_DQN curve (in blue) shows the temperatures of
the " simulated DQN " case of the first floor in the research
house, while Tin1_sim_BL (in blue) in Figure 8(b) shows
the temperatures of the simulated baseline case (i.e., fixedsetpoint
control) on the first floor. Note that, in both top
plots of Figure 8(a) and (b), the red Tin1_Tstat curve, which
represents the temperatures measured by thermostats during
the DQN deployment, is plotted as a reference to signify
the difference between the " simulated DQN " and baseline
cases. The " simulated DQN " case closely resembles the
DQN deployment because the two curves in the top plot of
Figure 8(a) are very close. Thus, the comparison between the
" simulated DQN " and baseline is highly credible. It is also
evident that the fixed-setpoint control gives quite different
results from the DQN deployment.
The mid plots in Figure 8(a) and (b) represent the second
floor of the research house and show similar patterns to the
first floor. The bottom plots show the simulated and measured
power use of the HVAC system. We may observe in
the bottom plot of Figure 8(a) that the measured power usage
during the DQN deployment (in red) matches well with the
power usage in the simulated DQN case (in blue). This also
table 4. The daily electricity cost and energy use comparison between DQN/RL and the fixed-setpoint baseline.
Baseline
DQN/RL
Date
Simulated
Cost (US$)
25 March 1.19
26 March 0.84
27 March 1.63
28 March 1.06
29 March 2.69
30 March 2.18
31 March 1.11
3.38
3.79
3.31
2.48
23.67
Simulated
Energy
(Wh)
10,475
5,005
13,886
9,766
1 April
2 April
3 April
4 April
All
52
19,316
14,412
13,453
29,741
31,228
24,237
16,872
188,389
ieee power & energy magazine
Simulated
Cost (US$)
1.18
0.64
1.29
0.97
2.23
1.59
1.08
3.02
3.7
3.31
1.87
20.9
Measured
Cost (US$)
1.47
0.73
1.35
1.17
2.09
1.57
1.11
2.82
3.47
3.04
1.82
20.64
Simulated
Energy
(Wh)
16,338
5,670
18,094
12,874
21,025
14,311
15,884
31,657
33,272
26,208
17,582
212,915
Measured
Energy
(Wh)
19,602
6,824
18,892
15,650
21,506
14,871
16,089
29,552
31,867
25,751
17,884
218,488
Cost Reduction
DQN/RL
Simulated
Versus Baseline
Simulated (%)
0.8
23.8
20.9
8.5
17.1
27.1
2.7
10.7
2.4
24.6
11.7
DQN/RL
Measured
Versus Baseline
Simulated (%)
-23.5
13.1
17.2
-10.4
22.3
28
16.6
8.4
8.2
26.6
12.8
may/june 2022
IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022
