IEEE Computational Intelligence Magazine - February 2022 - 37

and a fully-connected layer. The encoder f(x) maps the input
time series to a fixed-length vector, z. The encoder can be any
block of neural networks that process time series input data. We
employ a 1 D-CNN as the encoder in our study because of its
demonstrated ability to handle time series data [20]. The prototype
layer contains m prototypes, where the number of prototypes
is at least as large as the number of classes,
mC .$ The
goal of the prototype layer is to learn a set of vectors that are
positioned close to the encoded input data. Therefore, the
length of each prototype pi
is equal to the length of the encoded
input z. The prototype layer computes the squared Euclidean
distance between the encoded input z and each prototype,
.
dz p
the final score as (( d /))min
score of one means the prototype pi
ii
=y
Wa ,=
2 << To normalize these values, we then compute
ad
i =- i
2
2
((dd .)( )) A
is identical to the encoded
maxmininput
z, and zero means that they are completely different.
Finally, a fully-connected layer (FC) computes a weighted sum
of these scores
where W is a Cm# weight matrix. A
Softmax layer is then applied to compute the probability distribution
over C classes. The prototypes are not interpretable on
their own because the distance measure between prototypes and
observation is measured in a flexible latent space. The generator
transforms the encoded input z to its corresponding picture ypic
by reshaping the encoded vector z to a nn# matrix and
employing a 2 D-CNN. The advantage of including a generator
is that it can transform the learned prototypes into their pictorial
representations after training.
B. Optimization
PIP's objective is to simultaneously achieve both high accuracy
and high interpretability. To obtain this goal, we jointly minimize
the parameters of all model components using stochastic
gradient descent, similarly to work described by Gee et al. [18]
and Li et al. [19]. Let
Dx yyk
t
= {( ), ,} be a labeled time
T
t=1
pic
series dataset having k dimensions. Throughout the remainder
of the paper we use xt
to denote the set of k values observed at
time t. Here, T is the length of multivariate sequence
x ,{ ,, }yC1 f!
pic
L EM
11
++
=+
mm22
mm
ED MD
RD RD
HH
HH
(, )( ,)
(, )( ,),
(1)
where H represents the set of all trainable model parameters
and {, ,,EM 1mm mm=
cross-entropy loss defined as:
(, )()( )( )
ED
H =+ -=
!
yy yy
loglog
11
(( ), )xy D
(t) T
t
1
and M is the mean squared error of the generator, as follows:
(, )( ).
MD 1
n
H =!
/
((
), )xy D
()t T
t=1 pic
t
yy
picpic
2
and }2m represent the term ratios. E is the
/ tt (2)
is the true label, and yC1! f {, ,} is a
corresponding picture for the true class. The optimization
problem minimizes the following:
We introduce a regularizer, R ,1 that encourages each prototype
to be positioned close to at least one of the training samples.
Additionally, regularizer R2
cluster around one prototype:
ensures that similar inputs
RD
1(, )min pzj=1
H
=
RD
m
1
2(, )min zpi
=1
H = / [, ]
n
1
jm
! 1
/ 1![, ]
n
in
ij
m
ji
2
2
2
2
,
.
(4)
(5)
PIP learns the weights for the individual loss terms during
model training. For each of a specific number of epochs, we
adjust the ratio, ,m for one selected term,
, ! (, ,, )12EM RR .
The selected term is penalized if the corresponding value is
greater than a threshold
its ratio, m .,
than the threshold x by lowering the corresponding m .,
x . The term is penalized by increasing
Similarly, we reward the term if it performs better
This
method is inspired by game theoretic methods introduced by
Arora et al., which supports data-driven learning of the parameters
[22]. For our experiments, we set
x 01= .. We initialize all
..
p()
e = mmm12m
e
() U UUU()
e
E
e
()
()
M
e
()
e
()
()
e
()
e
e
U mmmm= +++ We update the selected m based on
its loss function value m, as follows:
()
e
()
E
e
()
M
e ee
()
()
.
m
() = )
e+1
me
me
()
()
e
e
(
1
+
-
() ,
1 ), i
-
-
,
,
()
()e
e
otherwise.
f , 1 x
(6)
We terminate this procedure after a specific number of
epochs in which L consistently decreases since indefinite
updates of sm could make the model training unstable. We
observe that updating sm for 20 consecutive epochs of decreasing
L values is enough to learn the
ms . Algorithm 1 details
the PIP training process.1
An important question is how to select the number of prototypes
to learn. Previous work [5], [18], [19], [23] specifies a
number of prototypes that is greater than the number of classes.
This allows the model to learn at least one prototype for each
class. When classes are complex, such detailed explanations can
be helpful. At the same time, users may be confused when a
single class is represented by multiple pictures. In earlier work,
anecdotal observations analyzed these tradeoffs. In our case, we
report results for alternative numbers of prototypes. To make a
final selection, we first consider the number of prototypes to be
equal to the number of classes. We then select the final number
of prototypes by increasing the number until performance (loss
minimization) converges.
While we demonstrate the learning process using greyscale
images, PIP can also learn colored prototypes. In cases containing
a large number of classes, colored input images will
further enhance interpretability of the blended prototypes
that PIP learns. Such colors can be selected by the user based
(3)
1Code: https://github.com/alirezaghods/PIPNet
m s to one for a fixed learning rate e # 05 At the end of each
epoch e, we choose a term to adjust during the next epoch.
Ratios m are selected with probability proportional to their
values {/ ,/ ,/ ,/ },
12
where
FEBRUARY 2022 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 37
https://www.github.com/alirezaghods/PIPNet
IEEE Computational Intelligence Magazine - February 2022

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2022
