IEEE Geoscience and Remote Sensing Magazine - December 2021 - 127

(a)
(b)
(c)
(d)
(e)
(f)
FIGURE 9. Abnormal displacement detection and correction. (a) The reference image. (b) The sensed image. (c) The displacement field
estimated by (3). (d) The aligned image overlapping (a). (e) The corrected displacement field. (f) The aligned image formed by overlapping
the corrected optical flow with (a) The highlighted road in (a) disappears in (b), leading to similar occlusion.
its accuracy for remote sensing image registration is relatively
low compared with the dense optical flow approach. In addition,
it is not vulnerable to LU-LC changes because it does
not have similar features for sparse optical flow estimation in
the changed region.
In summary, optical flow estimation has been developed
in computer vision for motion estimation in superresolution
reconstruction for several decades, whereas it is in the initial
stage of use in remote sensing image registration. Optical
flow estimation is a superior pixel displacement calculation
approach that is particularly interesting in the case of very local
deformation due to, for example, terrain elevation, which
has considerable influence on high-resolution image registration
[61]. The efficiency of optical flow estimation should be
considered when applying it to remote sensing because a wide
field of view (WFV) is a characteristic of remote sensing image.
Therefore, due to social development and seasonal changes,
LU-LC changes are frequent phenomena for multitemporal
remote sensing images. The dense optical flow approach is
sensitive to such changes, leading to abnormal displacement
and the alteration of the content of an aligned image. Therefore,
efficient and accurate correction should be integrated into
the initial optical flow estimation when used for registration.
FEATURE-BASED REGISTRATION
The feature-based approach directly exploits the abstract
features of an image, rather than the pixel intensity, for registration.
Feature refers to a distinct geometrical or advanced
characteristic extracted by a specified approach. Geometrical
features are distinct points, line segments, and closed boundary
regions in a remote sensing image that can be detected
or extracted by extant or novel approaches. Advanced features
are abstract descriptions of local regions, which are extracted
by a neural network (NN) (especially in the deep learning approach)
to represent the original image. Geometric features
are understood as being conventional for feature-based registration,
and the use of advanced features is defined as novel
feature-based registration.
CONVENTIONAL FEATURE-BASED METHOD
In general, salient and distinctive features, such as points,
line segments, and closed boundary regions, are manually
DECEMBER 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
Pixels
Feature Points
FIGURE 10. Sparse optical flow.
127
Optical Flow
and automatically detected to represent the original remote
sensing image. The feature correspondence is then established
between the reference and sensed images by a similarity comparison
of the feature descriptors. The geometric relationship
is calculated, guiding a sensed image that is spatially aligned
with the reference. Ultimately, coordinates in the sensed image
are transformed. The transformed coordinates are usually
noninteger, and they are calculated by interpolation to
acquire their intensity values, as demonstrated in Figure 11. In
the following, we summarize geometrical feature extraction
and matching because research into this subject has been at
the core of the traditional feature-based approach.
FEATURE EXTRACTION
The feature extraction mentioned here is a representation of
feature detection and extraction. Detection aims to locate distinctive
features in an image and determine their positions.
In the feature-extraction stage, the recognizable descriptor is
uniquely constructed, identifying the detected feature. Formerly,
features were manually selected. This approach is still
in use today, as in the " image-to-image registration " module
in Environment for Visualizing Images (ENVI) software. Experts
require a considerable amount of time for this approach,

IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021