IEEE Geoscience and Remote Sensing Magazine - December 2021 - 160

◗ Scene classification: Similar to image classification, one
image or patch is classified into a single label. However,
one scene is usually much larger than an image patch.
Hence, a different network architecture is required.
◗ Semantic segmentation: One image or patch is segmented to
a classification map of the same dimension. Training such
neural networks also requires densely annotated data.
◗ Object detection: This is much like scene classification.
However, detection often requires the estimation of the
object location.
◗ Registration/matching: This provides binary classification
(matched and unmatched) and estimates the translation
between two image patches. Such tasks require that pairs
of two different image patches be matched as training data.
IMAGE/SCENE CLASSIFICATION
So2Sat LCZ42
So2Sat LCZ42 [200] follows the local climate zones (LCZs)
classification scheme. The data set consists of 400,673
pairs of dual-polarity Sentinel-1 and multispectral Sentinel-2
image patches from 42 urban agglomerations, plus
10 additional smaller areas, across five continents. The image
patches are hand labeled into one of the 17 LCZ classes
[213]. The Sentinel-1 image patches contain a geocoded,
single-look complex image as well as a despeckled Lee-filtered
variant. In particular, it is the first Earth observation
data set that provides a quantitative measure of the label
uncertainty, achieved by letting a group of domain experts
cast 10 independent votes covering 19 cities. It therefore
can be considered a large-scale data fusion and classification
benchmark data set for cutting-edge ML methodological
developments, such as automatic topology learning,
data fusion, and the quantification of uncertainties.
OpenSARUrban
OpenSARUrban [199] consists of 33,358 patches of Sentinel-1
dual-polarity images covering 21 major cities in China. The
data set was manually annotated according to a hierarchical
classification scheme, with 10 classes of urban scenes at its finest
level. Each image patch has a dimension of 100 × 100 pixels,
with a pixel spacing of 10 m [the Sentinel-1 ground-range-detected
(GRD) product]. This data set can support deep learning
studies of urban target characterization and content-based SAR
image queries. Figure 10 shows samples.
SEMANTIC SEGMENTATION/CLASSIFICATION
SEN12MS
SEN12MS [202] was created based on its previous version
SEN1-2 [203]. It consists of 180,662 triplets of dualpolarity
Sentinel-1 image patches, multispectral Sentinel-2
image patches, and Moderate Resolution Imaging Spectroradiometer
(MODIS) land cover maps. The patches are
georeferenced, with a ground sampling distance of 10 m.
Each image patch has a dimension of 256 × 256 pixels. We
160
OpenSARShip 2.0
This data set [209] is based on its previous version, OpenSARShip
[210]. It contains 34,528 Sentinel-1 SAR image
patches of different ships, with automatic identification
system (AIS) information. For each SAR image patch, the
creators manually extracted the ship length, width, and
direction as well as the vessel type by verifying the data
on the Marine Traffic website [209]. Roughly one-third of
the patches are extracted from Sentinel-1 GRD products,
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE DECEMBER 2021
expect this data set to support the community in developing
sophisticated deep learning-based approaches for
common tasks, such as scene classification and semantic
segmentation for land cover mapping.
MULTISENSOR ALL-WEATHER MAPPING
The Multisensor All-Weather Mapping (MSAW) [204] data
set includes high-resolution SAR data, which covers 120 km2
in the area of Rotterdam, The Netherlands. The quad-polarized
X-band SAR imagery from Capella Space, with a 0.5-m
spatial resolution, was used for the SpaceNet 6 Challenge. In
total, 48,000 unique building footprints have been labeled
with additional building heights.
PolSAR IMAGE DATA SET ON SAN FRANCISCO
This data set [205] consists of PolSAR images of San Francisco
from eight different sensors, including Airborne SAR,
Advanced Land Observing Satellite (ALOS)-1, ALOS-2, RadarSat-2,
Sentinel-1A, Sentinel-1B, Gaofen-3, and Radar Imaging
Satellite (data compiled by E. Pottier of the Institute of
Electronics and Telecommunications of Rennes). Five of the
eight images were densely labeled to five or six land use land
cover classes in [205]. These densely annotated images correspond
to roughly 3,000 training patches of 128 × 128 pixels.
Although the data volume is relatively low for deep learning
research, this is the only annotated multisensory PolSAR data
set, to the best of our knowledge. Therefore, we suggest that
its creator increase the number of annotated images to enable
its greater potential use.
OBJECT DETECTION
MSTAR
MSTAR [207] is one of the earliest data sets for SAR target
recognition. It consists of 17,658 X-band SAR image chips
(patches) of 10 classes of vehicles plus one class of simple
geometric-shaped targets. The collected SAR image patches
are 128 × 128 pixels, with a resolution of 1 ft in the range
and azimuth. In addition, 100 SAR images of clutter are provided.
In our opinion, the number of image patches is relatively
low for deep learning models, especially considering
the number of classes. In addition, this data set represents a
rather ideal and unrealistic scenario: vehicles are centered
in the patch, and the clutter is quite homogeneous, without
disturbing signals. However, considering the scarcity of such
data sets, MSTAR is a valuable source for target recognition.
IEEE Geoscience and Remote Sensing Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2021
