IEEE Geoscience and Remote Sensing Magazine - June 2016 - 30

huge amount of computational burden when training the
model. Another way is to use a multiscale receptive field
size that can train filters with different sizes and generate
multiscale feature maps.
In the HDNN, the last layer's feature maps are divided into T blocks {B 1, B 2, f, B T} with filter sizes of
{s 1 # s 1, s 2 # s 2, f, s T # s T }, respectively. The ith block covers
ni feature maps of the final convolutional layer. Then the
activation propagation between the last two convolutional
layers can be formulated as
B t = v (C L - 1 * f t + b t),

(15)

where Bt denotes the tth block of the last feature maps, f t
denotes the filters of the corresponding block, and v denotes the activation function.
Having learned the multiscale feature representations
to form the final convolutional layer, an MLP network
is used to classify the features. The output of the HDNN
is a two-node layer, which indicates the probability of
whether the input image patch contains the target. Some
of the vehicle-detection results are referred to in [30],
from which it can be concluded that, although there are
a number of vehicles in the scene, the modified CNN
model can successfully recognize the precise location
of most of the targets, indicating that the HDNN has
learned fairly discriminative feature representations to
recognize the objects.
UNSUPERVISED METHODS
Although the supervised DL methods like the CNN and its
modified models can achieve acceptable performances in
target recognition tasks, there are limitations to such methods since their performance relies on large amounts of labeled data, while, in RS image data sets, high-quality images
with labels are limited. It is therefore necessary to recognize
the targets with a few labeled image patches while learning
the features with the unlabeled images.
Unsupervised feature-learning methods are models that
can learn feature representations from the patches with no
supervision. Typical unsupervised feature-learning methods are RBMs, sparse coding, AEs, k-means clustering, and
the Gaussian Mixture Model [104]. All of these shallow
feature-learning models can be stacked to form deep unsupervised models, some of which have been successfully applied to recognizing RS scenes and targets. For instance, the
DBN generated by stacking RBMs has shown its superiority
over conventional models in the task of recognizing aircraft
in RS scenes [105].
The DBN is a deep probabilistic generative model that
can learn the joint distribution of the input data and its
ground truth. The general framework of the DBN model
is illustrated in Figure 4. The weights of each layer are updated through layer-wise training using the CD algorithm,
i.e., training each layer separately. The joint distribution
between the observed vector x and the L hidden layers is
june 2016

ieee Geoscience and remote sensing magazine

DBM Structure

h3
RBM

Hidden Layers

h2
h1

Visible Layer

Directed
Belief Nets

fIgURe 4. The simple structure of the standard DBN.

P (x, h 1, h 2, f, h l) = ( % k = 0 P (h k h k + 1)) P (h l - 1, h l ) , where
P (h k h k + 1) is a conditional distribution for the visible units
conditioned on the hidden units of the RBM at level k, and
P (h l - 1, h l) is the visible-hidden joint distribution in the
top-level RBM. Some aircraft detection results from large
airport scenes can be seen in [105], from which we can see
that most aircrafts with different shapes and rotation angles have been detected.
l-2

sCene understandinG
Satellite imaging sensors can now acquire images with a
spatial resolution of up to 0.41 m. These images, which
are usually called very high-resolution (VHR) images, have
abundant spatial and structural patterns. However, due
to the huge volume of the image data, it is difficult to
directly access the VHR data containing the scenes of interest. Due to the complex composition and large number of land-cover types, efficient representation and understanding of the scenes from VHR data have become a
challenging problem, which has drawn great interest in
the RS field.
Recently, a lot of work in RS scene understanding has
been proposed that focuses on learning hierarchical internal feature representations from image data sets [50], [106].
Good internal feature representations are hierarchical. In
an image, pixels are assembled into edgelets, edgelets into
motifs, motifs into parts, and parts into objects. Finally, objects are assembled into scenes [107], [108]. This suggests
that recognizing and analyzing scenes from VHR images
should have multiple trainable feature-extraction stages
stacked on top of each other, and we should learn the hierarchical internal feature representations from the image.
UnsUPerVised hierarchicaL
FeatUre-LearninG-Based methods
As indicated in the "General Framework" section, there
is some work that focuses on unsupervised feature-learning techniques for RS images scene classification, such as
sparse coding [58], k-means clustering [60], [109], and topic
model [110], [111]. These shallow models could be considered to stack into deep versions in a hierarchical manner
[31], [106]. Here, we summarize an overall architecture of
31

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2016