IEEE Power & Energy Magazine - May/June 2022 - 21

formulation and its corresponding well-established problemsolving
routine, if there is any.
Today, when the metaheuristics community invents a
new algorithm, we often see it applied to estimate load forecasting
models. In load forecasting literature, we can also
find many papers that present hybrid algorithms, such as
GA + PSO + ANN, GWO + SVR, and GA + ACO + Fuzzy.
The claimed contribution is often twofold: 1) the combination
of these specific algorithms/techniques has never been
done before for load forecasting and 2) the accuracy from
the proposed hybrid algorithm is superior to its counterparts.
However, in the authors' experience, most, if not all of these
papers tend to be hard to read, impossible to reproduce, and
have no presence in the load forecasting tools used by power
companies. Although these papers may add load forecasting
as an indirect application to corresponding metaheuristic
algorithms, their contributions to load forecasting literature
appear to be marginal thus far. More about the accuracy
issue will be discussed in the " Illusion 5. The Highest Accuracy
Reported in the Academic Literature Represents State
of the Art " section.
Clarification 3
For most of the models in load forecasting literature, parameter
estimation can be formulated as an optimization problem.
Metaheuristics is one way to find sufficiently good
solutions to these optimization problems. However, combinations
or hybrid uses of different metaheuristics with other
AI techniques appear to offer a marginal contribution to the
load forecasting practice thus far.
Illusion 4: A Three-Layer, Feedforward
ANN With 35 Hidden Neurons Is Better
Than the One With 41
Researchers have tried many types of ANNs, such as feedforward,
recurrent, and deep ANNs. Among the various
network structures with different setups, three-layer, feedforward
ANNs, due to their relative simplicity, were most
frequently used before deep learning was brought to load
forecasting. Some papers argue that the superior performance
of an ANN structure is due to some minor changes,
such as increasing the number of hidden neurons from 35
to 41, increasing hidden layers from one to two, and changing
activation functions to radial basis functions. All of
these changes can be viewed as hyperparameter tuning. The
resulting setup of hyperparameters may work well for one
specific data set but fail for the others. The contribution to
load forecasting from such results tailored for a specific case
study is insignificant.
Nevertheless, this is by no means discouraging practitioners
from trying various ANNs or other AI techniques.
Instead of devoting resources and efforts to fine-tuning
hyperparameters, researchers have been encouraged to look
at the bigger picture. Here we list three research directions,
which are not meant to be exhaustive:
may/june 2022
✔ First, investigate novel methods, not a hybrid of different
metaheuristics, to efficiently and effectively tune
the hyperparameters.
✔ Second, find actionable insights, rules of thumb, and
practical guidelines to help forecasters better navigate
and understand hyperparameter tuning.
✔ Finally, explore new structures of ANNs and new
AI techniques that may benefit load forecasting. For
instance, the structures of the three generations of
ANNSTLF are quite different, which led to an improvement
in forecast accuracy.
Clarification 4
Given a network structure, such as three-layer, feedforward
ANNs, the " optimal " number of hidden neurons is unique on
a case-by-case basis. Attention should be paid to exploring
the innovative design of ANNs structures, such as recurrent
and deep neural networks.
Illusion 5: The Highest Accuracy
Reported in Academic Literature
Represents State of the Art
Many factors affect load forecast accuracy, such as data
quality, load composition, and size of the load. For example,
given all other factors set the same, the load forecast accuracy
of an independent system operator (ISO) is expected to
be higher than that of a power company within the ISO. The
accuracy reported in one paper could be for a specific area
during a certain period, which may not be generalized to
other jurisdictions.
Even putting aside various uncontrollable factors, there
may be a tendency for some authors to exaggerate forecast
accuracy as a result of flawed processes of model building
and forecast evaluation. For example, a paper may report
day-ahead forecast accuracy values in 2020 from a particular
model M based on a proposed methodology, which beats
three other benchmark models based on existing techniques
in the literature. All four models are built based on data from
2017 to 2019. Everything may look legitimate, fair, and rigorous
on paper. However, behind the scenes, the authors may
have tried 10 other variants by tuning hyperparameters of
the proposed methodology, among which model M had the
highest accuracy. The other variants may not be better than
the three benchmark models. In other words, the year 2020
is used for model selection without being disclosed to the
readers. This is known as peeking into the future in forecasting.
In reality, a forecaster cannot pick a model based on
perfect information of future load values.
Another approach used to promote a proposed model is
cherry-picking. Authors may secretly manipulate data without
disclosing details in the paper. Some observations may
be excluded from the model-building process, while others
may be replaced by different values. All of these changes
to the raw data have implications for final forecasts. In
other words, data preprocessing methods may have put the
ieee power & energy magazine
21
IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022
