IEEE Power & Energy Magazine - May/June 2022 - 27

yesterday, and a week ago ,,;(XX X
d
i
d
i
-17
i 1= for system
d
i
load, and i 2= for wind power generation). The bottom gray
square represents weekday dummies (D).
The purple circle is the predicted variable-here, it is
the price for hour 18, i.e., 6 p.m. A set of 24 such models
yields forecasts for all 24 h of day d. The " chessboards " to
the left illustrate the importance of selected explanatory
variables. The greener the square, the more often selected
the feature (vertical axis) for a given target hour (horizontal
axis; hour 18 is emphasized) in the test period. Dark
gray squares represent features that were (almost) never
selected. There is a correlation between yesterday's load
forecasts for hour h today and the price for this hour-
see the green diagonal in the middle " chessboard. " In
addition, yesterday's load forecasts for the morning peak
(hours 8-9) and late evening (hours 22-24) also carry relevant
information-see the green horizontal stripes in the
same " chessboard. "
Deeper and Deeper
In the past, the challenge of using NNs was computational
complexity. Training small structures-like the one in Figure
1(b)-was feasible, but calibrating larger models was
time-/resource consuming and, in some cases, even impossible.
For instance, during the optimization phase, the gradient
would often become either zero or infinity, leading to the
so-called vanishing gradient problem, effectively preventing
the weights from changing their values.
Over the last decade, advances in computational resources
the massive usage of GPUs and optimization algo(e.g.,
rithms)
have made it possible to efficiently train complex
structures, including NNs with hundreds of inputs, multiple
outputs, many hidden layers, and links to earlier layers (see
Figure 5). Because networks whose depth (i.e., the number of
layers) was not limited to just a single hidden layer systemically
showed better results and generalization capabilities,
80
70
60
50
40
30
20
10
pd−1,24
...
pd−7
DMon
Winter
Summer
DTue
...
DSun
Xd
...
D
pd,h
the field and models were named deep learning (DL) and
deep NNs (DNNs), respectively, to stress the importance of
depth in the achieved improvements.
While this success of DL models was initiated in computer
science applications, e.g., image and speech recognition
as well as machine translation, the benefits of DL also
spread in the late 2010s to energy-related applications, such
as EPF or wind power forecasting. Since then, deep models
have been heavily used to better exploit and model the
nonlinear relations between energy-based quantities (prices,
load, and generation) and their drivers (e.g., human behavior
and calendar effects).
In the context of EPF, early works were based on feedforward
architectures with features (i.e.,
inputs) modeled
as hyperparameters. The latter are settings of the models
(e.g., the number of neurons per layer, type of activation
pd−1,1
pd−1,2
pd−1
...
pd−2
Model Inputs
Model Output
30,000 40,000 50,000
Yesterday's Load Forecasts for Today (MW)
60,000 70,000
figure 2. The relationship between electricity prices
and the forecasted load is nonlinear and exhibits distinct
seasonal variations, as illustrated by the 2017 data from the
Scandinavian Nord Pool market.
may/june 2022
figure 3. A LASSO-estimated regression, the next step in
the evolution of EPF models. Gray squares denote vectors
of variables, and dashed arrows reflect the fact that
LASSO may eliminate some of the links. Like in Figure 1,
white squares represent inputs, and the purple circle is the
output variable. (Source: The human evolution graphic is
a modification of the " Human Evolution Scheme " by JoséManuel
Benitos. Licensed under the terms of the GNU Free
Documentation License.)
ieee power & energy magazine
27
Today's (Actual) Prices (€/MWh)
IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022
