IEEE Power & Energy Magazine - May/June 2022 - 38

The drawback of this approach is that it requires many
samples to build confidence in the correctness of the
network prediction, including prediction of the security
boundary. As illustrated in Figure 4, using only a small
number of samples could suggest that the boundary is predicted
accurately, although there may still be an undiscovered
mismatch. Related to this is that even for a larger number
of samples, an effective sampling strategy is difficult
to devise along the security boundary, especially if many
power system states are considered. A second shortcoming
of the traditional evaluation is that there are no guarantees
the neural network classification will not falsely change
between discrete samples.
Instead, we apply a new method that reformulates the
neural network to an optimization problem. By solving it,
we can certify how the neural network classifies entire input
regions (grey rectangles), i.e., as secure or not secure [Figure
4(b)]. The procedure works as follows:
✔ First we take a discrete operating point. This could
be a random sample from the training set or any other
operating point for which we know it is secure (the
process is very similar for points that are not secure).
✔ Then we solve an optimization problem to compute
the closest input for which the neural network prediction
changes from secure to not secure.
The distance between the reference point and this input
determines the size of the region around the reference operating
point where the classification is guaranteed not to
change. In other words, the result of this optimization supplies
us with the guarantee that all the points inside this
region will be classified by the neural network as secure.
This is represented as the colored rectangles in Figure 4 (i.e.,
each rectangle corresponds to a different region we certify).
Using that, the operator can start trusting the neural network
as they can now anticipate how it will behave for any possible
point in the certified regions.
The neural network is no longer a black box. We can
repeat this process for many reference points until we
obtain several regions with certified mapping and unveil
the neural network behavior across the input domain to system
operators.
This procedure can also be used to evaluate the neural
network robustness illustrated in Figure 4. For the point we
determined through our optimization, we can also compute
a ground-truth classification and assess whether the
neural network predicts the security boundary correctly.
Thus far, existing approaches could only evaluate misclassification
through random sampling. This is the first time that
a systematic procedure can measure how well a neural network
predicts a power system's security boundary. If we are
not satisfied with the network performance, we can add the
misclassified points to the training data set with the correct
ground-truth labels and retrain the neural network to improve
its robustness. This is a systematic and well-defined procedure
that can be repeated until we reach a desired level of
38
ieee power & energy magazine
robustness. The moment the operator has the performance certificates
and is satisfied with the neural network performance
is when we will see neural networks applied in safety-critical
power system applications, such as security assessment.
Worst-Case Guarantees: What Is the Worst
Neural Network Prediction Error?
Let us consider a guiding example where a neural network
acts as an optimal power flow algorithm. It predicts how
much controllable generators should produce for a given
load situation so that 1) the production cost is as inexpensive
as possible and 2) there is no violation of system constraints,
such as transmission line loading limits. Existing optimal
power flow algorithms can perform this task quite well.
However, the optimal power flow problem in its full nonlinear
form is still a major challenge to solve in a reasonable
time. Well-trained neural networks can determine a solution
up to 1,000 times faster and consider constraints that are
impossible to include directly in conventional optimization
methods (e.g., differential equations representing stability
constraints). Now imagine that we need to run thousands of
scenarios through an optimal power flow algorithm, considering
different load profiles and highly varying injections
from renewables, and determine the cost-optimal and safe
generation dispatch for all of them. That is when neural networks
can be quite helpful.
We have to keep in mind, however, that a neural network
does not necessarily predict a solution that respects all system
constraints and is also the least expensive option. Its
strength is its extremely fast computation, not necessarily its
accuracy. When testing the network, we want to ensure that
predictions 1) do not significantly violate constraints (e.g.,
do not breach more than 1% of the maximum limits) and 2)
remain close to the optimal solution from a cost perspective
(e.g., they determine a solution that does not cost more than
1% of the optimal cost).
Similar to classification neural networks, the standard
approach used to assess network performance after training
is to take all samples of the test data set, compare the
network prediction to the physical ground truth, and compute
whether and to what extent constraint violations occur
for specific samples (see Figure 5(a)]. As an additional
metric, we can also calculate the Euclidean distance of
the neural network prediction to the ground-truth optimal
point for these points to calculate the " optimality " of the
prediction. Reviewing all test data set samples, we can then
easily find the maximum of these violations and maximum
" optimality distance. " However, this process does not give
any guarantee that the worst prediction that can occur was
found. With conventional neural network assessment methods,
the only option for improving this is to evaluate the
metrics on even more data samples. Beyond a point, the
procedure becomes computationally expensive and still
does not yield certainty that an even worse prediction has
not been missed.
may/june 2022
IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022
