IEEE Geoscience and Remote Sensing Magazine - September 2022 - 332

PERSPECTIVES
RAHUL RAMACHANDRAN, MANIL MASKEY , CHRIS LYNNES,
ARUN JOHN, AND TATHAGATA MUKHERJEE
Investigating Different Data-Traceability
Approaches to Prevent Data Swamps
S
cience users typically obtain data from authoritative
or trusted sources referred to as repositories. An authoritative
source is defined as an entity that is approved by
a legal entity to develop or manage data for a specific purpose
[4], [5]. Different government agencies generating Earth
observation (EO) data are one example. An authoritative
source can also be an actively managed repository
of valid or trusted data that is recognized by a community
and supports data governance and stewardship practices.
One can also have secondary sources that serve as
trusted sources and provide trusted data [6]. A trusted
source can be a service provider or an organization that
publishes data obtained from a number of authoritative
sources. These publications are often compilations and
subsets of the data from more than one authoritative
source. They are " trusted " because there is an " official
process and/or agreement " for compiling the data from
authoritative sources, data are actively managed, and
the limitations, currency, and attributes are known and
documented. NASA mirroring Sentinel-1 data from the
European Space Agency is one such example of a trusted
source.
With the adoption of cloud computing, the notions
of authoritative and trusted sources of data are changing.
Cloud computing has rapidly become a viable approach
for building cyberinfrastructures that move data
out of institutional silos into a common computational
platform [1]. Cloud computing provides scalable, on-demand,
pay-as-you-go access to a pool of computing resources
and as such provides the building blocks needed
to make it easier, more efficient, and more economical
to configure data processing and analysis platforms for
scaled computing and reduced time to analysis. A data
lake is a core component in this new cloud-based analysis
paradigm where the data lake provides a flexible data
Digital Object Identifier 10.1109/MGRS.2022.3203865
Date of current version: 25 October 2022
332
store that organizes a wide variety of big data to address
data processing and analysis needs at scale. A data lake
supports multiple ways of data ingest and flexibility to
accommodate a variety of data storage based on volume
and a variety with different access patterns. It supports
data processing, including preprocessing and data transformations
steps, catalogs to support search and discovery,
and robust governance practices for complete data
lifecycle management.
In an ideal situation, there exists only one authoritative
or trusted data lake on a cloud platform curated with
EO data products from different agencies and organizations.
Users can then effectively utilize both the data
and cloud computing for data processing and analysis at
scale and be assured about the provenance of the data.
However, the reality is that we are not moving toward
this vision of a data lake. Each agency or data producing
organization is using their own cloud platform based
on their constraints, or some are still using in-house
infrastructures. As a result, many data products are being
replicated and mirrored at different cloud platforms
by third-party actors. For example, multiple copies of
many MODIS datasets (e.g., MODIS MOD13A1) with
different versions can be found on different commercial
cloud platforms. These third-party actors trying to help
other users of the data by mirroring data to a computing
platform as a bulk data transfer is both labor intensive,
costly, and time consuming. In some cases, the mirrored
product is also reformatted and/or restructured to improve
the ease and speed of analysis. However, these
third-party actors provide a varying level of data governance
and stewardship support for the mirrored data.
This situation of data duplication by third-party actors
without any data stewardship support is threatening to
create data swamps.
A data swamp, unlike a data lake, has little organization.
More critically, it lacks data curation with little
to no active data management throughout the data
IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE SEPTEMBER 2022
https://orcid.org/0000-0002-5087-6903
IEEE Geoscience and Remote Sensing Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - September 2022
