Publication Date: 2018-01-11

Approval Date: 2017-12-07

Posted Date: 2017-11-08

Reference number of this document: OGC 17-022

Reference URL for this document: http://www.opengis.net/doc/PER/t13-NA001

Category: Public Engineering Report

Editor: Guy Schumann

Title: OGC Testbed-13: NA001 Climate Data Accessibility for Adaptation Planning

OGC Engineering Report

COPYRIGHT

Copyright © 2018 Open Geospatial Consortium. To obtain additional rights of use, visit http://www.opengeospatial.org/

WARNING

This document is not an OGC Standard. This document is an OGC Public Engineering Report created as a deliverable in an OGC Interoperability Initiative and is not an official position of the OGC membership. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, any OGC Engineering Report should not be referenced as required or mandatory technology in procurements. However, the discussions in this document could very well lead to the definition of an OGC Standard.

LICENSE AGREEMENT

Permission is hereby granted by the Open Geospatial Consortium, ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.

THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD. THE INTELLECTUAL PROPERTY IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY.

This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.

Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications.

This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.

1. Summary

This Engineering Report describes all Testbed-13 activities relating to the Climate Data Accessibility for Adaptation Planning requirements of the National Aeronautics and Space Administration (NASA). It discusses relevant experiences made during implementation including recommendations to the sponsor, and provides resulting standards change requests to the appropriate working groups. Additionally, it develops best practices for data and model integration and serves as a guidance document to work with NASA Earth Science Data System (ESDS) working groups and externally provided data. The added value of this Engineering Report is to improve interoperability and to advance location-based technologies and realize innovations with regards to NASA Climate Data and NASA ESDS objectives.

Testbed 13 supports the ESDS Vision [https://earthdata.nasa.gov/earth-science-data-systems-program/about-the-esds-program/esds-mission] stated as:

"Make NASA’s free and open earth science data interactive, interoperable and accessible for research and societal benefit today and tomorrow".

The ESDS Program oversees the lifecycle of earth science data with the principal goal of maximizing the scientific return from NASA’s missions and experiments for research and applied scientists, decision makers and society at large. Testbed-13 results are applicable to this ESDS Goal:

"Ensure access to data and services that are useful and usable by a wide community of user."

To meet this goal, the Climate Data Accessibility thread in Testbed 13 sought to:

  • Improve climate data accessibility for the scientist and non-scientist.

  • Broaden climate adaptation essentials.

  • Collaborate with the Federation of Earth Science Information Partners (ESIP) [http://www.esipfed.org].

  • Work with the sponsor to continually improve components so that they are fit-for-purpose throughout Testbed-13 lifecycle.

A pre-Testbed-13 concept development study executed by OGC identified a number of datasets, portals, data centers, simulation models, and other web services suitable for addressing the stated goals of the Testbed-13 modeling thread. OGC have issued a Request for Information (RFI) to its membership to solicit interest, experiences, and data and model availability. In this context, OGC closely cooperates with ESIP to ensure knowledge transfer with ESIP partners. ESIP is an open, networked community that brings together science, data and information technology practitioners. ESIP’s mission is to support the networking and data dissemination needs of its members and the global Earth science data community by linking the functional sectors of observation, research, application, education and use of earth science. The OGC and ESIP have a Memorandum of Understanding (MoU) that promotes coordination between the two organizations on topics of common interest to facilitate effective knowledge transfer partnerships.

Testbed-13 participants were requested to help complete the set of available material. The overall goal was to gain experiences from the integration process, communicate the best practices and recommendations back to the Sponsor on potential improvements that will further simplify the integration process, understand interoperability issues arising from different formats, interfaces, protocols, and access policies. The Climate Data Accessibility for Adaptation Planning work package was demonstrated in a scenario defined in this document.

1.1. Requirements

Figure 1 illustrates the work items and requirements in the context of Climate Data Accessibility for Adaptation Planning.

Figure1
Figure 1. Climate data accessibility for adaptation planning requirements and work items.

The Testbed-13 deliverables described in this document meet the following requirements:

  1. Data integration: Demonstrate the integration and analysis of earth observation data made available at various portals (e.g. http://www.prepdata.org) and data serving systems in the context of use cases one and two described in the Deliverables section.

  2. Model integration: Integrate simulation models into the demonstration that can be parameterized and executed on demand through standardized Web Application Programming Interfaces (API) such as the OGC Web Processing Service (WPS) interface standard.

  3. Process improvements: All demonstrations implement the context described above. The work captures all experiences made during the integration process and derives best practices and recommendations for improvements that simplify workflows for scientists and non-scientists. This is done whilst considering account data, data access and data formats, models and model parameterization and execution, portals, access policies, and other elements that influence the integration experience.

1.1.1. Deliverables

The following list identifies the work items assigned to the requirements, note that work items are not exclusive to the work package described in this Engineering Report (ER).

  • NA101: Agriculture Scientist Client – This client component provides access to climate data, agriculture predictions and other data relevant for an analyst to assess the effects of drought on mass migration. The client supports the integration of on demand models through a WPS 2.0 interface.

  • NA102: Non-Scientist or Analyst Client – This client component provides access to climate data and other data relevant for an agriculture scientist to predict the effects of drought on crop production. This client also provides access to on demand models through the WPS 2.0 interface.

  • NA103: Prediction WPS – This component is made accessible through a WPS and enables access and control of predictive models relevant to drought and agriculture prediction based on climate and other data. The service supports different parameterization options to serve both scientists and non-scientists made accessible through the clients in NA101 and NA102.

  • NA104: WCS access to climate data – This Web Coverage Service (WCS) will provide access to NASA Climate data for use by the other components in Testbed-13. The climate data shall be made available in formats and resolutions appropriate for scientists and non-scientists.

The following use-cases shall be considered:

  • Use Case 1: Agriculture Researcher considering the effects of climate on crop production needs easy access to climate prediction/data for ingest into crop forecasting models, potentially requiring temporal and spatial sub-setting of climate model variables relevant to rain and soil conditions. Access to the right datasets, with effective system-to-system API, GIS based data formats (EG Shape, GeoTIFF) in addition to providing option to ingest subset HDF, NetCDF formats.

  • Use Case 2: Use by non-scientist or analysts to better understand climate impacts to populations by accessing climate model prediction data for ingest into local GIS based systems for population and critical infrastructure overlay.

These deliverables are designed to broaden distribution access and formats of climate reanalysis, climate model data, and climate based observational data for ease of system-to-system ingest, access (EG: API, WCS, other), and data delivery formats (File: HDF, NetCDF, Shapefile, GeoTIFF) for ingest by the scientist and non-climate scientist.

The two use cases for the system are represented by the two client work items, as discussed earlier. The main differences between the clients are as follows:

  • The amount of configuration required to produce a successful output.

  • The ability to upload manually configured files for running APSim, an Agricultural Production Systems sIMulator.

  • The constraints associated with the non-scientist client (i.e. the interface only offers a subset of the functionality to the user).

  • Contrarily, the freedom offered with the expert client to configure and execute the model without constraint.

The basis for the requirements is that many people needing climate reanalysis and climate model data (some cases observational data) do not have effective system-to-system access to climate model data and do not necessarily have geospatial expertise, i.e. understanding of file formats such as HDF, NetCDF and GRIB. They are also not necessarily climate scientists yet crop yield predictions based upon climate change are required for their work.

1.2. Key Findings and Prior-After Comparison

This section describes the status of discussion in the OGC Emergency & Disaster Management/Law Enforcement And Public Safety (EDM/LEAPS) Domain Working Group (DWG), identified as being most relevant for the addressed topic. This document is reviewed by the EDM/LEAPS DWG to ensure that the latest developments have been considered.

1.2.1. Key Findings Relevant to DSI Modeling Thread in Testbed-13

The flowchart of the DSI Modeling work package as illustrated in figures 1 and 2 develops full interoperability between geospatial data, models and their parameterization as well as visualization of data and model outputs. More specifically, with reference to NASA Climate Data and Accessibility, this Testbed-13 thread shows proper use of NASA data, primarily the Modern-Era Retrospective Analysis for Research and Applications (MERRA) version 2 dataset, for crop yield prediction. It also introduces geographic intelligence to the crop modeling process, something it is lacking at the moment for the particular model used.

Specifically, for the MERRA-2 data used, a key finding is that, although NASA data is freely and openly accessible to all, the MERRA-2 data, as also other NASA data, are scientific data that are often included in large structured files. Finding data in the large number of collections of files NASA holds proved to be very time-consuming. If the NASA (or other) cataloging systems supported searching for variables within datasets/collections that have specific units of measure, data location would have been less of a manual chore. Once found, the data files typically contain many variables and have their own variable naming which are oftentimes ambiguous. Moreover, when served in a web service setting using WCS 2.0, temporal subsetting proves difficult (WCS 2.1 addresses this, but the server will have to implement that optional feature). In this testbed, under the DSI Modeling work, the spatial subsetting of the MERRA-2 NetCDF file has been tested successfully for the Bay area. However, the temporal subsetting was not tested and such a test is recommended under the future work section. Other, less significant caveats, and also advantages of NASA HDF5-, NetCDF-, and GeoTIFF-formatted data are reported throughout this ER.

One of the key goals for this ER was to make valuable but complex datasets such as NASA’s MERRA-2 more accessible. This has been achieved by exploring the ease of access and performance of common clients. However, it remains a fact that datasets like the MERRA-2 are high-level scientific data and therefore require operations performed within a scientist client context. In fact, one of the recommendations for future work is that it should be explored how a scientist client could be used to predefine or preset the data visualization and simple parameterization for a non-scientist client and/or non-expert user.

1.2.2. Other Examples of Advancing in Academia and Science

The DSI Modeling work package for the Testbed-13 requirements NA001, NA101, NA102, NA103 and NA104 looked into how to efficiently couple data and numerical models such that they operate through web protocols and results as well as data visualized by clients. For instance, the National Science Foundation is already supporting similar activities in academia and science. The Community of Surface Dynamics Modeling Systems (CSDMS; http://csdms.colorado.edu/wiki) is establishing a web interface that lets users configure and operate a model. Although CSDMS mostly uses a web interface to run single or coupled models, they have experimented in the past with coupling data as input to models, and will explore this more in the coming 5 years. Setting standards in how models and data should be configured is of key importance to make coupling work and the DSI Modeling work package of Testbed-13 helps realize this step.

1.3. What does this ER mean for the Working Group and OGC in general

The purpose of the Emergency & Disaster Management (EDM) DWG is to promote and support the establishment of requirements and best practices for web service interfaces, models and schemas to enable the discovery, access, sharing, analysis, visualization and processing of information to the forecasting, prevention, response to and recovery from emergency and disaster situations. The mission lies in improving interoperability of geospatial products and other information consumables that can be shared across these communities. The following are the two main objectives for this ER:

  • Identify interoperability standards gaps and opportunities to support improved EMDR information sharing, collaboration and decision making.

  • Propose or encourage initiation of Interoperability Program studies, experiments, pilot initiatives, testbed threads or demonstrations to address technical, institutional and policy related interoperability challenges, and identify and engage the interest of potential sponsors for these activities.

The Law Enforcement And Public Safety (LEAPS) DWG promotes and supports the establishment of local, national, regional and international requirements and best practices for web service interfaces, data models and schemas for enabling the discovery, access, sharing, analysis, visualization and processing of information. This geospatial and temporal information is used comprehensively to address crime, terrorist activities and public safety incidents in an operationally effective way.

These two groups now form one DWG (EDM/LEAPS) given that their objectives and general purpose are very similar and overlapping for many applications, especially during disaster situations.

The objectives of this DWG are synergistic with the requirement and deliverables in the Testbed-13 Modeling Thread, i.e. to facilitate access and formats of NASA climate reanalysis, climate model data, and climate based observational data for ease of system-to-system ingest, access, and data delivery formats for ingest by the scientist and non-climate scientist. Many needing observations and model data do not have effective system-to-system access to models or data, however crop model predictions are still required for their work.

This situation is very common in the emergency response and disaster management sector, and hence the importance of this ER to the EDM/LEAPS DWG.

1.4. Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:

Table 1. Contacts
Name Organization

Guy Schumann

Remote Sensing Solutions, Inc.

Albert Kettner

UC Boulder, DFO, Observer through Remote Sensing Solutions, Inc.

James Gallagher/Nathan Potter

OPeNDAP

Sam Meek

Helyx SIS

Lei Hu

George Mason University

Dean Hintz

Safe Software, Inc.

1.5. Future Work

Details on Future Work are given in the final chapter of this ER. Below is just a short list of the most significant items recommended for future work:

  • WCS and/or clients should support some method of getting data at a position or set of positions over a time series without having to download the entire dataset or subset.

  • In light of large-scale dynamic process-based computer simulations, spatial and temporal subsetting of a NetCDF output from such models that produce a temporally and spatially dynamic result would be needed.

  • Test implementation of a transactional WPS (WPS-T), which would relieve burden on implementers to second guess requirements from users by providing a platform for them to execute their own processes at will.

  • In the context of complex scientific data and data formats, explore how a scientist client could be used to predefine data visualization and simple parameterization for a non-scientist client and/or non-expert user.

1.6. Foreword

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

2. Terms and definitions

For the purposes of this report, the definitions specified in Clause 4 of the OWS Common Implementation Standard OGC 06-121r9 shall apply. In addition, the following terms and definitions apply.

2.1. Abbreviated terms

  • API Application Programming Interface

  • APSim Agricultural Production Systems sIMulator

  • CSDMS Community of Surface Dynamics Modeling Systems

  • CU University of Colorado

  • EDM Environmental and Disaster Management

  • ER Engineering Report

  • EPA Environmental Protection Agency

  • ESDS Earth Science Data System

  • ESRI Environmental Sciences Research Institute

  • FME Feature Manipulation Engine

  • GIS Geographic Information System

  • GML Geographic Markup Language

  • GMU George Mason University

  • HDF Hierarchical Data Format

  • LEAPS Law Enforcement And Public Safety

  • MERRA-2 Modern-Era Retrospective analysis for Research and Applications, Version 2

  • MoU Memorandum of Understanding

  • NASA National Aeronautics and Space Administration

  • NetCDF Network Common Data Form

  • RFP Request For Proposals

  • RSS Remote Sensing Solutions, Inc.

  • TIE Test Interoperability Experiment

  • WCS Web Coverage Service

  • WPS Web Processing Service

  • WMS Web Map Service

  • WMTS Web Map Tile Service

3. Overview

This ER includes the following major chapters:

3.1. Summary Chapter

This chapter serves as introduction and background material to this ER and links to the LEAPS DWG and also the Testbed-13 Request for Proposals (RFP) document contents. It focuses on the DSI Modeling Component work, outlined in detail in the Components Chapter.

3.2. Components Chapter

This is the main chapter of this ER and includes detailed sections on the following Components and describes the connections between them as well as the respective TIEs. It also includes a brief scenario setting and framework of the demonstration video.

The different Components of the Modeling subthread are:

  • WCS from OPeNDAP: Providing NASA (MERRA-2) data to the WPS and all clients (scientist expert & non-scientist clients)

  • WPS from Helyx: APSim crop yield (agricultural) model process and implementation description

  • Scientist Clients: — Agriculture expert client from George Mason University (GMU) — FME expert client for Safe Software

  • Non-Scientist Clients: includes the following three clients: — Web GIS-style client from Remote Sensing Solutions (RSS/CU) — QGIS open-source desktop client from RSS/CU — FME client from Safe Software

3.3. Future Work and Recommendations Chapter

This chapter outlines general observations and caveats encountered and gives recommendations and suggestions for future work.

4. Components & Component Scenario

4.1. Setting the story

Agricultural yield prediction is a complicated process reliant on a variety of factors such as soil, relief, weather, crop type, and disease. Configuration of each of these variables and sub-variables requires knowledge beyond that of parties who could utilise the outputs from crop-yield predictions.

Rice crop/fields or grapes are interesting to simulate since these crops provide a compelling story about water usage and/or droughts. Rice was used as the main crop of interest for this testbed scenario since grapes require an extension module in the APSim prediction model which is beyond the scope of this sub-thread. Additionally, in the wider Bay area, Sacramento, Colusa, or Glenn counties have the bulk of California’s rice crop, and this crop is already included in the APSim model.

As an introductory part to the demonstration video, a short video (https://youtu.be/PHGRdovNGmM) and high-res satellite imagery of the area that includes rice fields (https://landsat.gsfc.nasa.gov/mapping-rice-managing-water/) have been selected.

The NASA video shown in the link below has also been used since it tells a compelling story about how satellite data and rice fields are being linked to support migration bird habitat.

https://phys.org/news/2015-09-satellite-migrating-birds-survive.html

For the story building, the United States Environmental Protection Agency (EPA) document accessible from the link below has been used to set the story. This is all very general and the focus is on rice crop, and the impact of drought/climate change.

https://www.epa.gov/sfbay-delta/what-are-challenges

Also, a lot of interesting content material that has been used can be found here: http://www.energy.ca.gov/2012publications/CEC-500-2012-033/CEC-500-2012-033.pdf.

4.2. Component Overview Chart

Figure 2 illustrates the connections and interoperability between the various component parts discussed in this ER.