IEEE Geoscience and Remote Sensing Magazine - June 2021 - 92
such as " dangerousness " or " scenicness, " which are generally
not visible in overhead images. With CNNs approaching,
and sometimes exceeding, human-level performance
for ground-level image interpretation, we can now consider
using the output of CNNs as a semantic description of a
given place and time. We can then use this description to
train a remote sensing model, which might take only satellite
imagery as input. In this way, we can extend the information
received from social media to areas where these
media are absent and, simultaneously, reduce the need to
manually annotate satellite images. This approach has been
applied for a variety of tasks, including for mapping scenes
categories [34] and for time-varying visual attributes [35].
However, there remain significant issues to address:
◗ The data to be included: Each source must add value, being
correlated to the task and independent from each other.
In an extremely multimodal setting, the volume and
velocity of data acquired from each source cannot be
directly controlled because they depend on completely
independent acquisition systems. Given that, it becomes
important to understand how the spatial coverage and
quality of each source vary. For example, clouds have
a significant impact on optical imagery but do not affect
synthetic aperture radar (SAR). Similarly, the coverage
and quality of various social media sources depend
on a variety of conditions, including their proximity to
tourist landmarks, population density, and differences
in the culture of a given social network (interested readers
are referred to discussions about social media data
quality in [36] and [37]). Therefore, we must choose
among not only satellite sources, but also social media
platforms, and must often apply further filtering, e.g.,
including only the data collected from cellphone cameras
[38] or only certain types of scenes [34].
◗ The quality of the matching between sources: Each source
must be georegistered to reduce spatial uncertainty during
training. The satellite/aerial-to-ground matching at
the scene level is highly challenging due to the large semantic
gap between the ground and overhead scene, but
it is still resolvable. For instance, Lin et al. [39] proposed
a dual-adversarial solution for an unsupervised satellite/
aerial-to-ground scene-adaptation solution. However,
it becomes very crucial when object-level matching is
concerned, e.g., when approaching automatic geolocalization
[40], [41], and, in particular, when considering
image synthesis [42]-[44], where strong geometric models
of the various modalities, with the ability to model
uncertainty, are strongly needed.
PERSPECTIVES
Considering social media data as extra sources for remote sensing
analysis is gaining momentum. Aside from classification
Query Location
Fusion
FIGURE 2. An example of a system that provides the likelihood of the presence of an object with multiple modalities. The ground and
satellite images provide hard and weak (picture-based) labels, respectively, to create a heatmap that shows the presence of objects in urban
areas. The location information is used to perform the fusion. (Sources: Google Maps; geoawesomeness.com; Wikipedia; Wikimedia.)
92
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE JUNE 2021
http://www.geoawesomeness.com
IEEE Geoscience and Remote Sensing Magazine - June 2021
Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - June 2021
Contents
IEEE Geoscience and Remote Sensing Magazine - June 2021 - Cover1
IEEE Geoscience and Remote Sensing Magazine - June 2021 - Cover2
IEEE Geoscience and Remote Sensing Magazine - June 2021 - Contents
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 2
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 3
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 4
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 5
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 6
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 7
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 8
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 9
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 10
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 11
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 12
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 13
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 14
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 15
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 16
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 17
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 18
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 19
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 20
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 21
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 22
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 23
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 24
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 25
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 26
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 27
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 28
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 29
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 30
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 31
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 32
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 33
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 34
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 35
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 36
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 37
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 38
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 39
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 40
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 41
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 42
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 43
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 44
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 45
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 46
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 47
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 48
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 49
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 50
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 51
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 52
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 53
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 54
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 55
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 56
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 57
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 58
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 59
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 60
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 61
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 62
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 63
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 64
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 65
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 66
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 67
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 68
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 69
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 70
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 71
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 72
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 73
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 74
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 75
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 76
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 77
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 78
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 79
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 80
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 81
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 82
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 83
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 84
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 85
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 86
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 87
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 88
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 89
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 90
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 91
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 92
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 93
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 94
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 95
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 96
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 97
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 98
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 99
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 100
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 101
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 102
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 103
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 104
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 105
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 106
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 107
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 108
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 109
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 110
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 111
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 112
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 113
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 114
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 115
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 116
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 117
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 118
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 119
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 120
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 121
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 122
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 123
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 124
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 125
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 126
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 127
IEEE Geoscience and Remote Sensing Magazine - June 2021 - 128
IEEE Geoscience and Remote Sensing Magazine - June 2021 - Cover3
IEEE Geoscience and Remote Sensing Magazine - June 2021 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2023
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2023
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2023
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2023
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2022
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2022
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2022
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2022
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2021
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2021
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2021
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2021
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2020
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2020
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2020
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2020
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2019
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2019
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2019
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2019
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2018
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2018
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2018
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2018
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2017
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2017
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2017
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2017
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2016
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2016
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2016
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2016
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2015
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2015
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2015
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2015
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2014
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2014
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2014
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2014
https://www.nxtbook.com/nxtbooks/ieee/geoscience_december2013
https://www.nxtbook.com/nxtbooks/ieee/geoscience_september2013
https://www.nxtbook.com/nxtbooks/ieee/geoscience_june2013
https://www.nxtbook.com/nxtbooks/ieee/geoscience_march2013
https://www.nxtbookmedia.com