IEEE Geoscience and Remote Sensing Magazine - December 2021 - 135

this transformation model is calculated with distinct tie features
in the plain region. With the transformation model,
rather than directly obtaining the aligned plain region, the displacement
guiding pixels to alignment is estimated. In mountainous
regions, the dense optical flow estimation borrowed
from computer vision is utilized to acquire the displacement
of each corresponding pixel. Then, the displacement fields
from different terrain regions are mosaicked (e.g., using the
inverse distance weighted function for uniform transitions in
image stitching) to obtain a seamless displacement field of
the entire image [177]. This is a creative combination of different
registration approaches in a coordinated way, differing
from the combined approaches mentioned in the " Registration
Based on the Combination Method " section with the
serial mode. Therefore, regional registration accommodating
complex geometric relationships that vary with terrain differences
may become a significant trend in remote sensing image
registration, giving full play to the registration advantages
of different approaches in various terrain regions.
HETEROGENOUS AND CROSS-SCALE
IMAGE REGISTRATION
Heterogenous and cross-scale images collected all at once
and at different times provide complementary information to
improve our understanding of an entire scene during Earth
observation or even during disaster rescues. However, such
data usually have dramatically different spatial resolutions,
intensities, noise, geometries, and so on, owing to different
imaging principles. Some studies have focused on spatial registration,
including optical image and SAR registration, optical
image and infrared image registration, and satellite image
and map registration [36], [57]. These works emphasized
the robust construction of descriptors to resist intensity and
noise differences and other influential factors. Large-scale
differences between cross-scale images (which are much
greater than four times the resolution difference between
the panchromatic and multispectral images) introduce difficulties
for extracting geometrical features from low-resolution
images that are similar to those from high-resolution
images. Thus, generating the tie features of cross-scale images
for transformation model construction, even during
high-precision registration, is difficult. Additionally, highefficiency
heterogenous and cross-scale image registration
remains an open problem that is worth researching in the
near future. For a concrete example, the approximately realtime
registration of optical and SAR images may offer an approach
for analyzing disaster regions as quickly as possible
for rescue purposes by means of registering and comparing
images before and after an event. These applications are vital
for rescue operations. Precise and efficient heterogeneous and
cross-scale image registration is a mandatory prerequisite for
high-precision, real-time applications.
SMART REMOTE SENSING IMAGE REGISTRATION
To register multiple remote sensing images, one simple and
conventional idea is to align them frame by frame, namely,
DECEMBER 2021 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
by converting multiple image registration into pair-to-pair
alignment. This process, learning from the simultaneous mosaicking
of multiframe images, specifies a reference image
connected to others and stitches other images to the reference
one. Therefore, when images to be registered are read into the
program, the coordinates of the four corners in each image
are extracted. The reference image is determined by comparing
these coordinates. As presented in Figure 14, images A,
B, C, and D are simultaneously aligned with the reference
image (marked in green) according to a general registration
strategy, as there is overlap between two images. Unlike frameto-frame
approaches, this technique needs to specify only the
reference image, and the intermediate results do not output
and input many times, which saves memory and improves
computational efficiency. From our point of view, this is
smart registration, which is particularly useful for WFV-image
generation. However, when images overlap, a more intelligent
approach needs to be developed.
Moreover, images to be registered may have small overlapping
areas. This overlap presents a challenge for high-accuracy
alignment because a small number of geometric and intensity
features is available for constructing the transformation
model. This problem should be intelligently solved to register
images with a low ratio of overlapping regions. Typically,
these images are used to produce WFV images by means of
stitching. Further solutions should be provided in the future.
Therefore, the large-scale, complex distortion of high-resolution,
heterogenous, and cross-scale remote sensing images must
be a focus of future research. In this situation, the traditional
single-registration approach may not meet requirements. For
real time, high-precision registration, a combination of alignment
approaches and high-performance computing is considered
very promising.
CONCLUSIONS
In this article, we presented a comprehensive and quantitative
summary of intensity-based, feature-based, and combined approaches
to remote sensing image registration. Conventional
methods and new applications of deep learning and optical
AB
Reference Image
C
D
FIGURE 14. The spatial position of multiple images to be registered.
135
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
