IEEE Geoscience and Remote Sensing Magazine - September 2019 - 45

CHALLENGES IN DATA REGISTRATION
Multisensor data registration has attracted increasing attention, but there are problems that need to be resolved. Considering the invariance of the semantic information of the
targets in multimodal images, the semantic feature or target
can be extracted for registration. Few works have been devoted to considering the complex cases involving scale, rotation, and affine issues in multimodal image registration.
Moreover, multisensor image registration based on CNNs is
a promising research direction.
HIGH-PERFORMANCE DATA PROCESSING
With large amounts of information, the complexity of processing algorithms, and the demand for fast response, the
time to automatically and efficiently process and deliver
SEPTEMBER 2019

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

0.3
0.25
Recognition Rate

REGISTRATION OF LIDAR AND OPTICAL IMAGES
Registration of lidar and optical images is common in
UAV-RS. The simple approach is direct georeferencing, but
it is difficult to achieve high-accuracy registration because
of platform vibration, unknown exposure delay, limitations of hardware synchronization and calibration, and
the low accuracy of onboard GPS/IMU sensors. There are
often three other strategies, as follows.
1) The problem can be considered as a multimodal image
registration by transforming lidar data into images, including grayscale-encoded height and return-pulse intensity images (also called reflectance images). Thus, areabased and feature-based multimodal image registration
can be used.
2) The problem can be converted to the registration of two
point sets, one lidar and the other image-derived. Iterative closest point (ICP) algorithms can be used. Salient
features are often extracted from two point sets for registration, used as the initialization of the ICP [152].
3) Registration can be performed directly between a lidar
point cloud and optical images, often based on line and
plane features.
With the first method, area-based techniques are often affected by the return-pulse intensity calibration, which determines the quality and correctness of the intensity image.
In contrast, feature-based methods provide robust registration [153]. Transformation error may also affect registration. In the second strategy, there is a large difference between the two point sets. Lidar provides a set of irregularly
distributed points with abundant information along homogeneous areas but poor information along object space
discontinuities; the image-derived point set is the opposite.
In addition, the accuracy of the image-derived point set and
the initialization of the ICP are nontrivial issues. As for the
third approach, it can be a daunting task to automatically
find conjugate features in both data sets.

remote sensing products to users has become an overarching concern for UAV-RS. One available approach is to perform data processing with low-complexity algorithms and
few manual interventions, such as image location estimation with fewer or no GCPs or direct georeferencing [102].
In the area of deep CNNs, some tricks for lightweight models have been proposed, including removing regions of proposal for object detection [154], model compression and acceleration by parameter sharing, pruning, low-rank matrix
decomposition, and knowledge distillation [155].
Another effective solution is high-performance computing [156], [157], such as parallel computing. Unlike serial

VIS-IR
CVC-Multimodal

0.2
0.15
0.1
0.05
0

SIFT

EHD

PCEHD
(a)

LGHD

RIDLG

0.7
SIFT
EHD

0.6
Recognition Rate

for development still exists. Moreover, it is challenging to
extract highly repeatable homonymy features from multiband images because of nonlinear radiometric differences.

PCEHD
LGHD

RIDLG

0.5
0.4
0.3
0.2
0.1
0
10°

60°

110°

160° 210°
Rotation
(b)

260°

310°

FIGURE 13. The results of visible and infrared image matching
(taken from Chen et al. [151]): (a) the average recognition rate of different multimodal image matching methods and (b) the recognition
rate of different rotations. These experiments were conducted on
the VIS-IR and CVC-Multimodal data sets. The recognition rate is defined as the number of correct matches among all of the correspondences. The results in (a) demonstrate the effectiveness of methods
based on structure information. However, most approaches provide
poor performance under rotation issues, as shown in (b). Thus,
there is still plenty of room for improvement. EHD: edge histogram
descriptor; PCEHD: phase congruency and EHD; LGHD: log-Gabor
histogram descriptor; RIDLG: rotation-invariant feature descriptor
based on multiorientation and multiscale log-Gabor filters.

IEEE Geoscience and Remote Sensing Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2019