IEEE Geoscience and Remote Sensing Magazine - September 2019 - 54

of the law [23]. For instance, deep learning can be used
to learn to control UAVs and teach them to fly in complex environments [248], [249]. We envision that UAVRS will be capable of automating the entire process,
from taking off to processing the data to executing proactive maneuvers. To this end, more issues need to be
considered, including intelligent perception of the environment, precision control, indoor/outdoor seamless
navigation, and positioning [250], [251].
DATA PROCESSING
The current level of data processing can satisfy most UAV
applications among remote sensing investigators. However,
problems remain regarding the need to facilitate data processing more automatically, efficiently, and intelligently,
thus improving UAV-RS's Earth-observation performance.
◗ Aerial view and path planning: One crucial and formidable issue is how to perform view and path planning to
ensure complete and accurate coverage of the surveyed
area with minimum flight time. UAV-RS often acquires
data either under manual control or using predesigned
flight paths, with the camera setting in a fixed direction, e.g., vertical or oblique. Consequently, it is difficult to perform complete and dense coverage, especially
in urban environments. One promising solution is to
take the initial scene reconstruction from the nadir acquisition as a reference to continuously optimize the
view and position [252]. An example of aerial view and
path planning is shown in Figure 19.
◗ Robust data processing: It is expected that UAV-RS will
be able to process remote sensing data of different
sources, qualities, resolutions, scales, distortions, and
so on, which is an imperative but challenging issue.
For instance, the technology should be adept at handling images that are water covered, cloud sheltered,
or of arbitrary attitude; photography loopholes; and
multisource images (close-range, low-altitude, and
oblique images or infrared and visible images) for aerial triangulation. Progress on these issues is anticipated
in the near future.
◗ Real-time/onboard data processing: Real-time or onboard
data processing plays a significant role in UAV-RS,

FIGURE 19. An illustration of aerial path planning in urban building
scenes (taken from Smith et al. [252]).

especially in time-critical remote sensing [253]. With
the wave of sensor miniaturization, it is expected that
FPGAs and GPUs will be designed to be light in weight,
low in energy consumption, and adaptable to miniaturized UAVs for onboard processing. In addition, the collected data should be processed based on high-performance computing, such as cloud computing.
◗ Deep learning for UAV-RS: Great success has been achieved
in image classification and target detection [254]-[257].
However, there is a great deal of room for deep learning applied in UAV-RS 3D geometric vision, especially
in image matching and pose estimation. Some critical
issues that should be investigated include the lack of a
large-scale annotation data set, weakly supervised learning for limited annotated data, and transfer learning for
off-the-shelf deep models.
◗ 3D semantic computing: There is a trend toward learning
to estimate 3D geometry and semantics jointly. More
geometric priors should be introduced to capture the
complex semantic and geometric dependencies of the
3D world. Another issue is the high memory consumption resulting from the need to store indicator variables
for every semantic label and transition, an issue that
should be researched [258].
◗ Information mining from UAV-RS big data: Data collected
from UAV flights can reach hundreds of megabytes per
hectare of surveyed area. Moreover, UAVs can form a
remote sensing network to provide fast, cloudless, centimeter-level, and hour-level data collection and accurate service on the Internet. This will inevitably generate
massive amounts of remote sensing data. Knowledge
mining from massive and heterogeneous remote sensing data is a great challenge. Deep learning and cloud
computing shed light on this issue. Also, optimizing
data acquisition to ensure complete and accurate coverage with minimum data volume and redundancy is crucial for reducing the difficulty of information mining.
APPLICATIONS
With advances in UAV platforms and remote sensing sensors, there is a potential for wider applications. Attention
may shift from monitoring Earth environments to human
and social understanding, such as individual/group behavior analysis and infectious disease mapping [259]. UAV-RS
also holds potential for the autonomous driving community. UAVs are employed to extend the perception capabilities
of a vehicle by using a small quadrotor to autonomously
locate and observe regions not visible to the vehicle and
detect potentially unsafe obstacles, such as pedestrians or
other cars [35]. More applications are on the way.
CONCLUSIONS
Compared to conventional platforms (e.g., manned aircraft
and satellites), UAV-RS presents several advantages: flexibility, maneuverability, efficiency, high spatial/temporal
resolution, low altitude, and low cost, among others.
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

SEPTEMBER 2019

IEEE Geoscience and Remote Sensing Magazine - September 2019

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2019