IEEE Geoscience and Remote Sensing Magazine - December 2021 - 74

DEEP FEATURES
Deep feature representation based on the layer-wise learning
of image patterns is a very promising research direction
for change detection in VHR images [66], [67]. Differing
from traditional handcrafted features, higher-level abstractions
(both linear and nonlinear features) can be automatically
extracted and optimized by multilayer neural
networks, which can retain crucial variations and discard
uncorrelated differences for change detection tasks [68]. In
recent years, many deep learning methods have been developed,
such as autoencoder (AE) models and convolutional
neural networks (CNNs), for deep feature extraction in
change detection with VHR images.
The AE is an unsupervised feature learning model that is
constructed by minimizing the reconstruction error. However,
it may learn a useless feature representation, such as a
simple copy of the input [69]. To overcome that issue, variant
models, e.g., the denoising AE (DAE) [70], sparse AE (SAE)
[71], and Fisher AE (FAE) [72], have been employed for VHR
change detection, with denoising, sparsity, and Fisher discriminant
criteria, respectively. Specifically, a stacked DAE
was used to learn high-level features from the local neighborhood
[70]. In [70], it was found that the filters learned by a
stacked DAE have a stronger representation capability than
existing explicit ones. Based on the SAE, Su et al. [71] transformed
a difference image into a suitable feature space for
suppressing noise and extracting key change information in
the change detection framework. Liu et al. [72] used the FAE
for unsupervised layer-wise feature learning and showed that
the model can generate more discriminative features than
the original AE. In addition to unsupervised feature learning
through the optimization of certain criterions, AE-based
models can learn effective features in a supervised way by
considering label consistency, e.g., the contractive AE [73].
It is well recognized that CNNs are effective in extracting
mid- and high-level abstract features by interleaving
convolutional and pooling layers [74]. According to the
feature learning strategy, CNNs can be categorized as unsupervised
[67], [75], [76], supervised [77], fine-tuning
[66], and transfer learning based [78]. For example, Zhan
et al. [75] used a pretrained CNN to automatically extract
deep spatial-spectral features for change detection in VHR
satellite images. Saha et al. [67] developed unsupervised
deep CVA for change detection, and a network trained on
remote sensing aerial images for semantic labeling by Volpi
and Tuia [79] was adopted for deep feature extraction.
As detailed in Figure 5, the experimental results demonstrated
that, compared to object-based methods, deep features
are effective for capturing change information and
are promising for distinguishing multiclass change information.
Wang et al. [77] trained a model through manually
selected samples, where the parameters of the shared
convolutional layers were initialized by the pretrained
ResNet-50 model, and the others were randomly initialized.
Hou et al. [66] chose to extract CNN-based deep
features through a fine-tuned Visual Geometry Group
74
(VGG)-16 by transferring a model pretrained on largescale
natural images to the remote sensing domain via an
aerial image data set. Liu et al. [78] proposed a CNN-based
transfer learning method for change detection. In particular,
the loss function was designed by combining high-level
features extracted from a pretrained model (i.e., the U-net
model trained on an open source data set) and semantic
information contained in change detection data sets.
Notably, deep learning methods depend on an enormous
amount of training data, which may not be available
for multitemporal VHR remote sensing imagery [74]. Meanwhile,
great differences in spectral properties and image
contexts among natural red-green-blue (RGB) images and
remote sensing data result in deep features extracted by finetuned
models that do not fully represent the essential characteristics
of remote sensing images. As a result, the contrast
between a small number of remote sensing data sets and
a large number of natural images during model learning
may hamper the further improvement of VHR change detection
using deep features. In recent years, large multitemporal
data sets have been released, such as 86 image pairs
from the DigitalGlobe satellite constellation (i.e., QuickBird,
WorldView-1, WorldView-2, and GeoEye-1) [80], 291 pairs of
multitemporal aerial images [81], and more than 700,000
labeled instances for building damage assessment [82]. It
can be anticipated that more and larger multitemporal VHR
remote sensing data sets with diverse image characteristics
and various acquisition conditions will appear in the near
future. In this case, the essential change features for VHR
remote sensing images can be effectively extracted by a deep
network specialized for multitemporal remote sensing data.
OBJECT-BASED FEATURES
Object-based features refer to spectral, geometry, texture,
extent, and contextual information at the object scale rather
than single pixels and groups of pixels within a kernel
filter/moving window. In this way, an image object is viewed
as the processing unit for change detection. An object is a
set of spatially adjacent pixels that are spectrally similar and
that can be extracted through image segmentation. Overall,
object-based features are effective in VHR change detection
since they mitigate radiometric differences, spectral variability,
and misregistration errors [38], [83]. However, appropriate
segmentation parameters, which are often dependent
on subjective and laborious trial-and-error experiments,
need to be determined [84]. Furthermore, shortcomings and
problems in different multitemporal image segmentation
strategies, e.g., 1) the segmentation of only one monotemporal
image, 2) the segmentation of stacked multitemporal images,
and 3) the independent segmentation of multitemporal
images, should be carefully considered and tackled [5], [85].
Specifically, geometric changes (e.g., the size and shape)
cannot be captured by 1) and 2) [85]. Moreover, strategy
2) may also result in " sliver objects " caused by image misregistration.
As for strategy 3), spatial correspondence
between multitemporal objects needs to be established.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
