Systems, Man & Cybernetics - July 2017 - 18
by additively combining the four decoding filter outputs
scaled with the magnitudes of select h values.
(a)
Constrained AEs for Enhanced Data Understanding
Neural networks are known for building hierarchical
models, but they do so in twisted ways that are difficult
to understand [17]. Attempts have been made to reconcile the interpretability of the autoencoding network
[13] and solve a long-standing open problem of understanding neural networks' processing. The emergence of
parts-based representation can be conceptually tied to
the nonnegativity constraints. Borrowing this concept
from biological analogies, one way to foster understandability in autoencoding is to constrain the encoder and
decoder's weights to be nonnegative. This allows easier
human interpretation, since the cancellation of terms in
the scalar products summation is eliminated [17]. In
practice, the cancellations are discouraged rather than
eliminated to ensure good accuracy. Enhanced data
understanding can result when the input data can be
decomposed into parts in each layer while the weights
are constrained to be nonnegative and sparse, as shown
in [13]. This imposition of nonnegativity constraint can
be incorporated into the SAE cost function in (7) in the
form of a penalty term and the resulting L 1 /L 2 NCSAE
[12], [13]. The decay term in (8) is replaced with (9). It
must be noted that the constraint imposed using (9) is a
(b)
Figure 4. The RFs or weights of a randomly selected
32 out of 196 (nl = 196) hidden neurons of (a) an
NNSAE and (b) an NCAE trained using the MNIST data
set. Black pixels indicate negative weights, gray pixels
indicate zero-valued ones, and white pixels indicate
positive ones. The range of the weights is scaled to
[−1, 1] and mapped to the graycolor map. w = −1 is
assigned to black, w = 0 to gray, and w = 1 to white.
MNIST Handwritten-Digits Characters
W1
tive
ep
Rec
s
Field
W
2
Dec
odin
g Fi
lters
Test Sample
≈ h1∗
(Hidden Activations)
+ h34∗
+ h48∗
+ h50∗
[h1, ..., h70] = [0.4, 0.0, 0.0,0.0,..., 0.0, 0.6, 0.0,0.0,..., 0.9,0.0, 0.5,0.0,0.0,..., 0.0,0.0]
Figure 5. A representation of a test image as a linear combination of four out of 196 constrained RFs and
decoding filters learned from the MNIST data set using an NCAE with a linear output activation function. The
input consists of 784 values corresponding to a 28 × 28-pixel image. Only 70 RFs with the largest activations to
test image 6 and their corresponding decoding filters are shown. The RFs and the decoding filters are rescaled
and portrayed as images on the right side. Black pixels indicate negative weights, and white pixels indicate
positive ones. The range of the weights are scaled to [−1, 1] and mapped to the graycolor map. w = −1 is
assigned to black, w = 0 to gray, and w = 1 to white. The biases are not shown.
18
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Ju ly 2017
Table of Contents for the Digital Edition of Systems, Man & Cybernetics - July 2017
Systems, Man & Cybernetics - July 2017 - Cover1
Systems, Man & Cybernetics - July 2017 - Cover2
Systems, Man & Cybernetics - July 2017 - 1
Systems, Man & Cybernetics - July 2017 - 2
Systems, Man & Cybernetics - July 2017 - 3
Systems, Man & Cybernetics - July 2017 - 4
Systems, Man & Cybernetics - July 2017 - 5
Systems, Man & Cybernetics - July 2017 - 6
Systems, Man & Cybernetics - July 2017 - 7
Systems, Man & Cybernetics - July 2017 - 8
Systems, Man & Cybernetics - July 2017 - 9
Systems, Man & Cybernetics - July 2017 - 10
Systems, Man & Cybernetics - July 2017 - 11
Systems, Man & Cybernetics - July 2017 - 12
Systems, Man & Cybernetics - July 2017 - 13
Systems, Man & Cybernetics - July 2017 - 14
Systems, Man & Cybernetics - July 2017 - 15
Systems, Man & Cybernetics - July 2017 - 16
Systems, Man & Cybernetics - July 2017 - 17
Systems, Man & Cybernetics - July 2017 - 18
Systems, Man & Cybernetics - July 2017 - 19
Systems, Man & Cybernetics - July 2017 - 20
Systems, Man & Cybernetics - July 2017 - 21
Systems, Man & Cybernetics - July 2017 - 22
Systems, Man & Cybernetics - July 2017 - 23
Systems, Man & Cybernetics - July 2017 - 24
Systems, Man & Cybernetics - July 2017 - 25
Systems, Man & Cybernetics - July 2017 - 26
Systems, Man & Cybernetics - July 2017 - 27
Systems, Man & Cybernetics - July 2017 - 28
Systems, Man & Cybernetics - July 2017 - 29
Systems, Man & Cybernetics - July 2017 - 30
Systems, Man & Cybernetics - July 2017 - 31
Systems, Man & Cybernetics - July 2017 - 32
Systems, Man & Cybernetics - July 2017 - 33
Systems, Man & Cybernetics - July 2017 - 34
Systems, Man & Cybernetics - July 2017 - 35
Systems, Man & Cybernetics - July 2017 - 36
Systems, Man & Cybernetics - July 2017 - 37
Systems, Man & Cybernetics - July 2017 - 38
Systems, Man & Cybernetics - July 2017 - 39
Systems, Man & Cybernetics - July 2017 - 40
Systems, Man & Cybernetics - July 2017 - 41
Systems, Man & Cybernetics - July 2017 - 42
Systems, Man & Cybernetics - July 2017 - 43
Systems, Man & Cybernetics - July 2017 - 44
Systems, Man & Cybernetics - July 2017 - 45
Systems, Man & Cybernetics - July 2017 - 46
Systems, Man & Cybernetics - July 2017 - 47
Systems, Man & Cybernetics - July 2017 - 48
Systems, Man & Cybernetics - July 2017 - Cover3
Systems, Man & Cybernetics - July 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202310
https://www.nxtbook.com/nxtbooks/ieee/smc_202307
https://www.nxtbook.com/nxtbooks/ieee/smc_202304
https://www.nxtbook.com/nxtbooks/ieee/smc_202301
https://www.nxtbook.com/nxtbooks/ieee/smc_202210
https://www.nxtbook.com/nxtbooks/ieee/smc_202207
https://www.nxtbook.com/nxtbooks/ieee/smc_202204
https://www.nxtbook.com/nxtbooks/ieee/smc_202201
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com