IEEE Power & Energy Magazine - May/June 2022 - 36

function parameters and neural network size, in theory, the
network has the power to approximate any possible function
or process.
The process of determining these parameters is commonly
referred to as the training of a neural network. It is a
part of a general ML algorithm, together with the creation of
the training data set beforehand and the testing carried out
afterward. Figure 3 illustrates this procedure and highlights
where new elements of neural network verification and physics-informed
neural networks enter the process.
Before neural network training, we start by putting together
a data set with both actual measurements and simulation
results (if the measurements are not enough). This data set
is usually a representative scenario, yet it is a small fraction
of the possible scenarios that could occur. It contains
both the inputs and the outputs, e.g., what is the operating
point and whether it is " safe " or " unsafe? " The goal is that
through this data set, the neural network learns the relationship
between inputs and outputs so that when training
is over, it can correctly estimate the output for any unseen
input not included in the training data set. To assess how
well the neural network predicts that output, we use a separate
test data set to statistically determine the neural network's
performance. The test data set is often a smaller
part of the one generated for training but kept aside to use
only for testing.
The next step in the ML algorithm is to determine the
neural network's parameters so that predicted outputs match
the true ones from the training data set as closely as possible.
The procedure is an iterative optimization algorithm, where
we pass batches of data several times through the neural
network until we arrive at a good match between network
outputs and those in the training set. When we are satisfied
with the performance on the training data set, or we reach a
predetermined time limit, we fix the parameters.
After training comes testing. Depending on how accurate
the neural network's predictions are on a random set
of previously unseen test samples, we conclude how " good "
the network is compared to others. But what happens if the
samples do not reflect reality? What if the neural network
achieves 99% accuracy but the test set fails to include critical
cases or even some high-risk outliers? How can an operator
trust this performance index? We address this in the next
section by describing methods that do not depend on the test
data set to determine the network performance.
The creation of a neural network is purely based on data
and statistical learning. One could even say that training a
neural network is an advanced form of nonlinear regression.
Physical models have thus far not been involved in training.
In traditional ML algorithms, a physical model influences
only the data-generation stage, either because data were
generated through simulations using first-principles models
or the data were collected from the actual physical process.
The physical models, in this case, are represented in a discrete
form through data points but not in equations. In the
36
ieee power & energy magazine
following sections, we explain how we adjust this process
to add the physical models inside the neural network training
in the form of continuous equations. But first, we focus
on how to obtain rigorous performance guarantees that are
not based on statistical indices and are not dependent on the
quality of the test set.
Guarantees of Neural Network Behavior
Thus far, conventional methods evaluate how well a neural
network performs by measuring its performance on a test set
(e.g., accuracy and other statistical indices). However, this
process is purely statistical and cannot tell us with certainty
what the prediction will be for any points not included in
this set, which is crucial for any safety-critical applications.
The goal is to develop techniques that evaluate whole continuous
regions instead of just discrete points when testing a
neural network. This allows us to consider any possible point
within an input region and guarantee the neural network performance
for any point in this region. We achieve that using
optimization methods. As it turns out, we can rewrite the
equations that define a neural network such that they can
be incorporated into an optimization problem. Due to this
reformulation, we can now analyze a neural network in a
completely new way.
Methods such as the ones mentioned in this article can
provide system operators with guarantees about how a neural
network will behave for entire regions of power system
operation. At the same time, they eliminate the dependency
on the quality of the test data set. Instead of having to sample
a potentially extremely large number of test data to cover all
possible scenarios (and still not extract any guarantee or provide
any certainty), we can now solve a single optimization
problem for one continuous part of the input region.
To illustrate this framework, we examine the following
two vital questions, one each for a classification and regression
task:
1) How large is the continuous input region for which
the neural network classification remains the same?
The answer provides a guarantee that any input in
this region will be classified to a specific known
class. Let us consider a power system. Such a guarantee
can state that the all operating points where
generator one is between 0 and 200 MW, generator
two is between 0 and 100 MW, and the load is between
0 and 300 MW will be classified as " safe " by
the neural network.
2) What is the largest prediction error of the neural network
across a continuous input region (regression)?
Considering power systems, assume a simple example:
we have two generators serving a load over two
transmission lines; we train a neural network to output
the combination of generator set points that result in
the minimum cost 1) for a load that varies between
10 and 300 MW and 2) without violating the transmission
capacity of any line. A worst-case guarantee
may/june 2022
IEEE Power & Energy Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Power & Energy Magazine - May/June 2022
