IEEE Geoscience and Remote Sensing Magazine - December 2015 - 19

and serializing for 20 rows can be performed in approximately 50-60 milliseconds (ms) on a data frame of approximately 60 MB containing 1,440,000 rows of data,
and approximately 160-180 ms on a data frame of approximately 240 MB containing 5,760,000 rows of data.
This latency is minimal for generating a data preview for
users, considering the amount of data filtering taking
place. This solution also scales to a large volume of files
and users through efficient use of computer memory by
only loading the data for a brief period of time, and also
through efficient use of HTTP caching.
The LP DAAC is using an approach to maintain portability and extensibility while adding new functionality
to the SDK tool box by using open-source modules and
capabilities such as Celery, JSON, and MongoDB. This approach can be implemented for both interactive and asynchronous client applications, reducing the level of effort to
add new capabilities.
8. ConClusions
Previous methods of accessing metadata from archives,
typically contained within text files such as XML are fairly
straightforward, and can be accomplished through simple
web interfaces within reasonable response times of a few
seconds. This type of access, however, is limited to obtaining information about the data and its attributes, but not
always about data quality, usefulness, or applicability.
Using OPeNDAP and other middleware services, the LP
DAAC has demonstrated accessing this information and
analyzing the actual data interactively has not only become possible, but can be performed in an uninterrupted
work session.
Continuous technological improvements are enabling
data archives to rapidly increase in size, depth, breadth,
scope and complexity at exponential rates. New methods
of data discovery, data mining, data visualization, and visual communication of data analysis are required to empower users to mine the wealth of information contained
in these archives.
Though wide area networks continually increase in
speed and bandwidth, they remain inadequate for user demands of increasingly complex information derived from
rapidly expanding data archives. Efficient data structuring
and services are required to provide interactive creation of
higher-level, derived-information products.
The LP DAAC implementation of OPeNDAP and webenabled middleware services attempts to leverage new data
access and delivery protocols to provide more efficient archive data access methods. User-defined temporal and spatial parameters can be used to aggregate or sub set archive
components, and middleware services condense the data
into the least amount of data possible to deliver the information desired by the user.
This new model for delivery of derived information is
dramatically more efficient by physically locating the data
retrieval and processing on systems near the data producer
december 2015

ieee Geoscience and remote sensing magazine

instead of systems controlled by the data consumer. The
processing and interaction occurs almost entirely on large
servers hosting the data, or systems connected to data
servers with high-bandwidth, low-latency local networks.
Transport of the requested information is pushed across
wide-area, low-bandwidth networks only after data are reduced in size by orders of magnitude.
By providing interactive access to Analysis Ready Data
with web services, the LP
DAAC can release scientists
from the burden of being data
using this weB seRViCes
managers. Satellite data usmodel, tAsks PReViouslY
ers can focus on their science
PeRfoRmed BY humAns
research instead of spending
in dAYs, weeks And
valuable time and resources
downloading, managing, and
months weRe PeRfoRmed
processing raw data. Using
in An AutomAted
this web services model, tasks
enViRonment in seConds,
which previously performed
minutes And houRs.
by humans in days, weeks
and months were performed
in an automated environment in seconds, minutes and hours.
OPeNDAP service layer access to the LP DAAC MODIS
archive has been implemented. Access to MODIS archives
via the Beta release of AppEEARS was made available to the
public September 29, 2015, at https://lpdaacsvc.cr.usgs.gov/
appeears. Current capabilities are implemented for a sub set
of the entire MODIS product suite. The services implemented enable expansion to accommodate all MODIS products,
additional science use cases, or additional data archives in
the future.
The description of current capabilities in this article
encompasses progress to date. Future plans incorporate
refinement of the current SDK and addition of new capabilities as user requirements evolve. User feedback will be
routinely solicited to drive new development. Also, application and service performance is continually monitored
and analyzed. Performance improvement strategies will
be implemented whenever reasonable, with the goal of
creating an interactive experience for users of LP DAAC
remote sensing data.
ACknowledgments
This work was performed under USGS contract
G15PD00766 for LP DAAC. Any use of trade, product, or
firm names is for descriptive purposes only and does not
imply endorsement by the U.S. Government.
Role of funding souRCe
All work was funded by NASA. Contract work was performed under NASA contract NNG14HH33I.
AuthoR infoRmAtion
Brian N. Davis received a Computer Science degree from Augustana University, Sioux Falls, SD, in 1985. Since 1986 he has
19

https://lpdaacsvc.cr.usgs.gov/

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2015