IEEE Power & Energy Magazine - May/June 2022 - 28

function, dropout rate, learning rate, regularization coefficient,
and choice of optimization algorithm) that need
to be optimized and tuned separately from the main NN
parameters (i.e., the weights). Like hyperparameters, the
features need to be selected/optimized separately from
the main parameters. Therefore, these early works proposed
DNNs where the hyperparameters and inputs were
optimized together.
The most prominent example of deep NN for EPF is
probably the DNN model proposed in 2018, whose inputs
and hyperparameters are optimized and tailored for each
case study without the need for expert knowledge. It uses
one of the simplest deep architectures proposed for EPF:
feedforward with two hidden layers, 24 outputs (i.e., it
jointly predicts 24 hourly prices), and the same inputs as the
LEAR model. The DNN's structure is identical to the one
in Figure 5(b), except for the feedback connections (dotted
arrows). In 2021, its open source code was released, and
the DNN model itself was recommended as a challenging
benchmark-even more than the LEAR model-for contemporary
EPF.
In terms of training and real-time usage, the DNN con3
6
9
12
15
18
21
24
Today's
(Predicted) Prices
3 6 9 12 15 18 21 24
3
6
9
12
15
18
21
24
Today's (Predicted) Prices
36 91215182124
Monday
Wednesday
Friday
Sunday
Today's (Predicted) Prices
36 91215182124
figure 4. The variable selection in the LEAR model calibrated to data from the Nord
Pool power market.
28
ieee power & energy magazine
D
pd−1
pd−2
pd−3
pd−7
Xd
1
Xd−1
1
Xd−7
1
Xd
2
Xd−1
2
Xd−7
2
pd,18
sists of two phases. In the first, the hyperparameters and
input features are jointly optimized using historical data. To
do this, the inputs are modeled as binary hyperparameters,
which can be either selected or discarded. This step is performed
periodically but not very often, e.g., once per month.
In the second phase, the DNN is recalibrated daily with the
optimal set of inputs and hyperparameters to account for
the newest market data. Since each hyperparameter/feature
selection yields a different local optimum, the DNN
can greatly benefit from averaging forecasts, that is, training
multiple DNNs and building a
forecast as the arithmetic average
of the individual predictions of
the DNNs.
The first wave of DL models
was followed, in the last few years,
by hybrid architectures utilizing
so-called long short-term memory
and/or convolutional NNs. Unlike
feedforward networks, the long
short-term memory architecture
has multiple feedback connections
and can process not only single
data points but also entire time
series. Convolutional NNs are regularized-to
prevent overfitting-
versions of multilayer feedforward
networks that use convolution
(instead of matrix multiplication)
in at least one of their layers.
However, despite the recent
work on DL models, it is unclear
whether all of the extra complexity
brings improvements in forecasting
accuracy. The existing
evidence indicates that, although
simple DL models (like a DNN)
can, on average, improve upon
LASSO-estimated regression
models (like LEAR), the performance
of more complex DL
models is generally unknown. A
common pattern that many works
in DL share is the failure to compare
the proposed models with
state-of-the-art statistical methods
and/or employ long enough
data sets to derive statistically
sound conclusions.
may/june 2022
Weekdays
Yesterday's Load Forecasts for Today
Yesterday's Prices
IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022
