IEEE Geoscience and Remote Sensing Magazine - September 2019 - 77

P(v, h) =

exp (- E(v, h))
,
H
NH

K

E (v, h) = - /

NW

/ /

k = 1 i, j = 1 r, s = 1

(1)
NV

h kij W krs v h, d - / c i

k

NV

k=1

i=1

u k ) v) - / c i
= - / h k $ (W

i=1
NV

/

NV

/

i, j = 1

v ij -

i, j = 1

K

NK

k=1
NK

i, j = 1

v ij -

K

/ bk /

/ bk /

k=1

h kij ,

h kij
(2)

u k ) v)ij + b kh
P ^h kij = 1 ; v h = sigmoid ^(W

P ^v ij = 1 ; hh = sigmoid ed / W k ) h k n + c i o .
k-1

(3)

ij

The conditional probability can then be redefined using the
max-pooling method:
P ^h ki, j = 1 ; v h =
P ^t = 0 ; vh =
k
a

1+

exp ^f ^h ik, j hh
/ exp _ f_ h kil,jlii

(il, jl ) ! b a

1+

/

(il, jl ) ! b a

1
,
exp _ f _ h kil, jlii

u k * v) ij,
f ^h ki, j h _ b k + (W

(4)

where t ka is a binary unit in the pooling layer connecting
block a in the detection layer, and b a _ "^i, j h: h ij belongs
to the block a. , pooling range.
The CDBN method provides a symmetrical connection between the detection and visible layers using shared weight. The
shared weight reduces the complexity of the computation time
and space. However, the weight can still possess some redundancy for RSIs because of the large amount of data. We propose a weight-compression strategy based on the deep-compression method [170] to improve original CDBNs and call it
dc-CDBNs. The new approach modifies the CDBN framework
by embedding the deep-compression technique, which performs network pruning, creates a sparse matrix, and produces
weight quantization and sharing. Weight quantization and
sharing are achieved using the k-means cluster method.
In the dc-CDBN method, the deep weight compression
includes three processes:
◗ pruning the network connections with weights that fall
below the given threshold and creating a spare matrix
for a connections index
◗ further compressing the pruned connections network
by weight quantization and a sharing mechanism to reduce the number of weights (weight quantization and
sharing implements multiple connections to share the
same weight using k-means clustering)
◗ compressing the quantization value with less data loss
using Huffman coding.
The quantization and sharing of weights ensures that
the same weights occur in the trained connections network
SEPTEMBER 2019

l

argmin /

i, j = 1

where H is a partition function, h = i + r - 1; d = j + s - 1;
u k is a horizontally and vertically
c i and b k are bias terms; and W
k
W
.
flipped array of
A conditional distribution of the convolution operation
of the block Gibbs sampling is as follows:

k=0

and that all weights are in the cluster. The subweights are
defined as W = " w 1, w 2, w 3, f, w m ,, and the clusters are
divided into i cluster centers, with linear initialization
z = " z 1, z 2, z 3, f, z l , (i % m). The minimum sum of
squares in a cluster is defined as follows:

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

z

/

i = 1 W ! zi

; W - z i ;2.

(5)

The compression ratio R dc of the quantization is given by
R dc =

nt ,
mn + it

(6)

where i clusters require m = log 2 ^i h bits for index encoding and i shared connection weights in a network with n
layer connections, where each connection represents t bits.
The minimum sum of squares represents multiconnections
sharing the minimum number of weights. The compression
ratio R dc represents the performance of the connections of
the shared weights during the quantization process.
To verify the effect of the improved algorithm, we extracted the UIS of the city centers of Pavia, Italy, and Nanjing,
China, using dc-CDBNs and then compared the operating
effects of the SVM, CNN, and CDBN algorithms, as shown in
Table 2 and Figure 10. In the experiment, we used a three-layer convolution operation and embedded deep compression
in the first layer. We found that the convolution operation of
dc-CDBNs can effectively extract the spatial features and edge
information. These features combine the spectral features of
images into a multifeature extraction strategy. In our study,
the compression ratio was set at R dc = 4. The Pavia city center data set consists of hyperspectral RSIs with 103 spectral
bands and a 2.5-m spatial resolution; the Nanjing data set
consists of Landsat 8 OLI images with a 30-m resolution.
The dc-CDBN method can achieve 93.93% and 94.55%
overall accuracy and enhancements of 4.35% and 7.07% for
the Pavia city center and Nanjing, respectively, compared
with the previously noted techniques. In terms of computational time and space efficiency, the dc-CDBN process reduces computation time by 19.2% and 23.4% and conserves
computing space by 24.5% and 26.1% for the Pavia city
center and Nanjing, respectively, compared with the CDBN
algorithm. As displayed in Table 2, the overall accuracy of

TABLE 2. A CLASSIFICATION PERFORMANCE COMPARISON
OF DC-CDBNs AND SVM, CNNs, AND CDBNs.
METHODS

DATA
SETS

EVALUATION
INDEX

SVM

CNNs

CDBNs

DC-CDBNs

Pavia
city
center

OA (%)

86.12

90.48

92.13

93.93

AA (%)

83.36

91.27

91.39

92.61

Kappa

0.8067

0.8895

0.9076

0.9156

Nanjing

OA (%)

84.38

88.07

92.86

95.51

AA (%)

83.15

87.31

91.74

93.84

Kappa

0.8354

0.8776

0.9178

0.9318

77
IEEE Geoscience and Remote Sensing Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2019
