IEEE Geoscience and Remote Sensing Magazine - June 2016 - 28

whole image, a simple feature extraction is conducted for
each proposal or the whole image to extract the low-level
descriptors that are invariant to shift, rotation, and scaling,
to some extent, such as SIFT [98], Gabor [99], and the histogram of oriented gradients (HOG) [97]. Next, the middlelevel feature representations can be generated by performing codebook learning on the learned descriptors. This
step is not essential, but using these low- or middle-level
features usually outperforms merely using the raw pixels
when learning hierarchical feature representations by the
following deep neural networks.
The deep neural networks such as the CNNs, sparse AEs,
and DBNs are hierarchical models that can learn high-level
feature representations in the deep layers automatically
generated by the features learned in the shallow layers.
Having learned the discriminative and robust representations of the proposals, a classifier such as an SVM is trained
with training samples composed of the representations of
some data and the corresponding supervisory information.
When a new proposal is generated from a new image, this
framework can automatically learn the high-level features
from the raw image, and then classification is undertaken
by the well-trained classifier to tell whether the proposal is
the target or not.

efficient location method and robust classifier for target
recognition in complex environments. In the literature, Cai
et al. [91] showed how difficult it is to segment aircraft from
the background, and Chen et al. [30], [92] made great efforts in vehicle detection in HR RS images.
The performance of target recognition in such a complex
context relies on the features extracted from the objects. DL
methods are well suited for this task, as this type of algorithm can extract low-level features with a high frequency,
such as edges, contours, and outlines of objects, whatever
the shape, size, color, or rotation angle of the targets. This
type of algorithm can also learn hierarchical representations from the input images or patches, such as the parts
of the objects that are compounded by the lower-level features, making recognition of RS targets discriminative and
robust. A number of these approaches achieved state-ofthe-art performance in target recognition by use of a DL
method [30], [48], [49], [52], [56], [93]-[96].

samPLe seLection ProPosaLs
To choose the most accurate area that exactly contains the
target, a number of proposals should be extracted from
the input image. Each proposal is usually a bounding box
covering an object that probably contains the target. The
most satisfactory case is that the target is in the center of
the bounding box, and the bounding box can just cover the
edge of the object.
There are different ways of selecting the proposals. The
baseline technique is the sliding window method [100],
which slides the bounding box over the whole image with
a small stride to generate a number of proposals. The sliding window technique is accurate and will not miss any
possible proposals that may exactly contain the target,
yet it is slow and burdens the subsequent feature-learning

Linear
Coding

Selecting
Proposals

RS Image

Deep Networks

Preprocessing

GeneraL deeP-LearninG FrameWorK oF
remote sensinG tarGet recoGnition
The DL methods used in target recognition can be divided
into two main categories: unsupervised methods and supervised methods. The unsupervised methods learn features from the input data without knowing the correlated
labels or other supervisory information, while the supervised methods use the input data as well as the supervisory information attached to the input to discriminatively
learn the feature representations. However, both of these
DL methods are utilized to learn features from the object
images, and the learning processes can be unified into the
same framework, as depicted in Figure 3.
The RS images are first preprocessed to subtract the
mean and divide the variance, or to simply convert the images to gray images with only one channel. Other preprocessing techniques compute the gradient images [97] of the
original image with a certain threshold [30]. The second
term of this general pipeline is extracting the object proposals, which is a bounding box locating the probable targets.
Following the process of selecting the proposals from the

Low-Level
Descriptor

Middle-Level
Features

High-Level
Features

fIgURe 3. A general framework of target recognition using DL methods. The high-level features learned by the deep networks are sent to
the classifiers to be classified (or directly classified by the deep networks for a supervised network).
june 2016

ieee Geoscience and remote sensing magazine

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2016