I. Executive Summary
A disaster, whether anticipated or not, can be an overwhelming event. Each disaster has its own unique details and progression, which can generate and cascade into additional crisis situations. The preparation and coordination of scalable responses can mitigate these challenges. Disaster Pilot 2021 brings the technological pieces together in order to reduce information preparation time and accelerate the ability to transform data from observation into decision. This will require bridging the divides between providers, responders and other stakeholders, forming a connected ecosystem of data and technologies, and developing the capacity to produce DRI (Decision Ready Information) products that answer decision makers’ questions almost as fast as they can be posed.
In order to successfully implement and test critical technical and organizational aspects of the envisioned ecosystem, the Pilot focused on scenarios addressing hazard early warning, monitoring, vulnerability assessment, disaster detection, impact awareness, and disaster response related to:
Landslide / flooding hazards and pandemic impacts within the Rímac and Piura river basins in Peru;
Flooding hazards and pandemic impacts within the Red River basin in Manitoba, Canada; and
Integration of Health and EO data and services for pandemic response the greater New Orleans area in the United States.
The Pilot was developed through stakeholder consultation in multiple workshops and meetings. It included DRI recipes that integrated earth observations with social, economic, health, infrastructure, environmental, and other information to address key indicators of disaster risk, vulnerability, and impact with the goal of directly influencing the scope and nature of disaster and pandemic response activities.
Further Pilot activities investigated getting this right information into the hands of the right people on the ground. For response practitioners, this involved generation of secure, portable Geopackage archives for distribution in connectivity-challenged circumstances. For the public, the Pilot explored Web publication of disaster information Web pages including linked structured data in order to connect well-known local geography with up-to-date conditions, observations, and predictions in common Web searches.
II. Keywords
The following are keywords to be used by search engines and document catalogues.
ogcdoc, OGC document, JSON
III. Preface
It is possible that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.
Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed upon by any implementation of the standard set forth in this document and to provide supporting documentation.
IV. Security Considerations
The overall aspect of security, authenticity, integrity, and confidentiality are tightly coupled, not only in emergency and disaster management situations. Ensuring trust in data being used for disaster management is a major concern. In terms of establishing trust into the data via security capabilities provided with online communication (e.g. HTTPS) does not apply, as the security mechanisms must be attached to the data itself to support the essential offline use. In OGC Testbeds since #15, this is defined as Data Centric Security.
The efficient and effective management of disaster impacts relies on many factors, including the interoperable access to relevant data provided by different stakeholders in existing Spatial Data Infrastructures (SDIs). The relevant data that otherwise might not be accessible, must be made available seamlessly to all identified actors of disaster management ensuring security and trust. The OGC GeoDRM Abstract Specification defines this ability as overriding access policies by pushing the red button.
But, enabling the “red button override” must not necessarily result into the loss of authenticity, integrity, and confidentiality of the data. As a stakeholder, it is important to be able to limit the further use of the data released for the purpose of disaster management. Constraints certainly involve the role of actors, also time and spatiotemporal constraints: who, when, and where shall access to the data for a designated disaster area be allowed? Extending the concepts of GeoXACML and Data Centric Security, explored in OGC Testbeds #15, #16 and the ongoing #17, helps to define solutions for maintaining security when packaging encrypted data for controlled use with verifiable trust as it is key for the actors.
During normal operations, system security controls guarantee GDPR compliance regarding personal data. But once the “red button override” is active, GDPR relevant consent buttons might countermeasure the effectiveness of responders. One typical solution to make data quickly available for disaster response management is to drop access control and put in place extra logging and provenance recording. But for online access, this approach introduces the threat of data leaking and can therefore not be used. Also, disabling access control would also have implications to data privacy and ethics.
Linking SDIs with Health infrastructures is a new emerging concept that is extremely relevant since the COVID-19 impact demonstrated its importance. Ethical concerns and data privacy are extremely relevant, but how can this be achieved under disaster management circumstances? From a technical perspective, it would potentially be too difficult for each system that provides disaster management data to be updated with an interoperable mechanism that supports the “red button overwrite”. Instead of trying to change existing APIs and their security controls, the concept was developed during Disaster Pilot ’21 where stakeholders produce a secure, trustworthy container of data that ensures integrity and confidentiality but also supports trust. The proposed solution is based on Secure GeoPackage that can be produced by stakeholders and then shared with relevant actors.
V. Submitting Organizations
The following organizations submitted this Document to the Open Geospatial Consortium (OGC):
- Secure Dimensions GmbH
VI. Submitters
All questions regarding this document should be directed to the editor or the contributors:
| Name | Organization | Role |
|---|---|---|
| Andreas Matheus | Secure Dimensions | Editor |
| Tony San José | SATELLOGIC | Contributor |
| Ignacio Correas | Skymantics | Contributor |
| Dave Jones | Stormcenter | Contributor |
| Zhe Fang | Wuhan University | Contributor |
| Simone Giannecchini | Geosolutions | Contributor |
| Paul Churchyard | HSR | Contributor |
| Guy Schumann | RSS-Hydro | Contributor |
| Dean Hintz et.al. | Safe Software | Contributor |
| Omar Barrilero | SatCen | Contributor |
| Josh Lieberman | OGC | Contributor |
| Pedro Gonçalves | Terradue | Contributor |
| Fabrice Brito | Terradue | Contributor |
OGC Disaster Pilot 2021 Engineering Report
1. Scope
This report is a summary of activities undertaken in the execution of Disaster Pilot 2021. These included integration and transformation of EO provider data using hybrid applications-to-the-data EO cloud exploitation platforms to seamlessly bring analysis-ready imagery, in situ, social, economic, environmental, health, and other data streams into scalable cloud environments where advanced processing, modeling, and algorithms can be directly and flexibly applied to them. They also included assessing and validating ARD standards, then integrating both space-based and local data sources on demand, providing targeted decision ready information products to local analysts and field responders through modern convenience API’s, optimized hybrid-cloud services, and mobile-ready online-offline Geopackage tools. Additional activities included Web search optimization for disasters through Web publication of linked “structured data” connecting well-known local geography with up-to-date conditions, observations, and predictions. Pilot activities focused on pandemics, flooding, and landslide disasters in the Piura and Rímac river basins of Peru, the Red River basin of Manitoba, and the greater New Orleans area.
2. Terms, definitions and abbreviated terms
This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.
This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.
For the purposes of this document, the following additional terms and definitions apply.
2.1. Terms and definitions
2.1.1. application
A self-contained set of operations to be performed, typically to achieve a desired data manipulation, written in a specific language (e.g. Python, R, Java, C++, C#, IDL).
2.1.2. application package
A platform independent and self-contained representation of an Application, providing executables, metadata and dependencies such that it can be deployed to and executed within an Exploitation Platform.
2.1.3. compute platform
The Platform providing the computational resources for the execution of the Application.
2.1.4. container
A container is a standard unit of software that packages up code and all its dependencies so that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
2.1.5. coordinate reference system
coordinate system that is related to the real world by a datum term name (source: ISO 19111)
2.1.6. exploitation platform
An on-line system made of products, services and tools for exploitation of data.
2.1.7. portrayal
presentation of information to humans (source: ISO 19117)
2.1.8. LiDAR
Light Detection and Ranging — a common method for acquiring point clouds through aerial, terrestrial, and mobile acquisition methods.
2.1.9. DCS
Data Centric Security
2.1.10. KMS
Key Management System
2.1.11. Decision Ready Information Component | D111-3 | Secure GeoPackage Service (aka GeoPackage Encryption Service)
Web Service extension to WMS 1.1 and WFS 2.0 to produce GeoPackages with encrypted content upon request
2.2. Abbreviated terms
API
Application Programming Interface
COG
Cloud Optimized GeoTIFF
COM
Component Object Model
CORBA
Common Object Request Broker Architecture
COTS
Commercial Off The Shelf
CWL
Common Workflow Language
DCE
Distributed Computing Environment
DCOM
Distributed Component Object Model
DCS
Data Centric Security
DEK
Data Encryption Key
EGS
Encrypted GeoPackage Service
EO
Earth Observation
GDAL
Geospatial Data Abstraction Library
IDL
Interface Definition Language
JSON
JavaScript Object Notation
KEK
Key Encryption Key
KMS
Key Management System
REST
Representational State Transfer
STAC
SpatioTemporal Asset Catalog
URL
Uniform Resource Locator
3. Abstract
This OGC Disaster Pilot ’21 (DP21) Engineering Report summarizes work done in the Pilot to increase disaster awareness among a range of disaster management stakeholders. Pilot participants implemented components of a data flow ecosystem to leverage analysis-ready earth observations and other datasets (ARD) and produce decision ready indicators (DRI) according to collaboratively developed workflow recipes. DP21 focused on the hazards of flooding, landslides, and pandemic, as well as the interactions and complications between them, in three regions including the Piura and Rimac river basins in Peru; the Red River Basin in Manitoba, Canada; and the greater New Orleans area in Louisiana, United States. The Pilot also prototyped providing information to field practitioners in secure geopackage formats, as well as leveraging linked data and structured web page information to optimize public web searches for disaster information.
4. Introduction
When a disaster strikes it can generate additional crisis situations. These can be overwhelming, but preparation and coordination of scalable response capacity can meet the challenges. Disaster relief forces from supporting jurisdictions can quickly integrate and analyze vast streams of realtime data from multiple sources to monitor the evolving situation and plan appropriate responses. Hybrid scalable cloud-based systems bring advanced AI processing, machine learning algorithms, and predictive models right to where earth observation and other data is already uplinked, prepared, and curated, generating analysis-ready data (ARD) with the characteristics, scale, and speed that the complex situation demands. Convenience API’s, as well as download-and-go geopackage containers and viewers, bring decision-ready information (DRI) directly to field workers’ mobile devices even in resource-constrained, low-connectivity areas. Meanwhile, web publication of “structured data” that link well-known content with up-to-the-minute situation observations enables search engines to push disaster-relevant information up in search results and help the public stay on top of fast-moving events. All of this is possible through the preparation of technologies, geospatial standards, and data sharing arrangements that bring the right information at the right time to the right people in the right place.
Following the success of Disaster Pilot 2019, the OGC Disaster Pilot 2021 brought all the above mentioned technologies together. Though elements had been explored and tested in various scenarios, it is OGC’s unique position in the center of the geospatial marketplace that allowed a single pilot to explore and test key state-of-the art technologies in an unprecedented combination. The Pilot was able to serve as an accelerated market research activity, while at the same time demonstrating the integration potential native to OGC Innovation Program activities. The emphasis was on testing end-to-end information flow related to the most urgent phases of disaster management, with an emphasis on first responders and other end users. The Pilot focused on four essential elements.
Scalable exploitation of earth observation data on hybrid cloud platforms using application-to-the-data technologies.
Development and implementation of recipes integrating and transforming ARD into DRI to provide specific guidance for decision and actions during disaster preparation, response, and recovery.
Generation of secure GeoPackage offline containers to facilitate taking all relevant information into the field.
Capability for structured data to enable web search engines to push disaster-relevant information up in search results during an ongoing disaster.
This work supported a vision of disaster response in which disaster relief forces are able to rely on information drawn from many EO and other data sources that can be integrated and analyzed quickly using the scalable capacity of hybrid, interoperable cloud platforms, specific adaptable recipes for generating decision ready indicators, and the power of models to distill relevant information into specific predictions, for example, flood risk.
The Pilot focused on scenarios addressing hazard early warning, monitoring, vulnerability assessment, disaster detection, impact awareness, and disaster response related to:
Landslide / flooding hazards and pandemic impacts within the Rímac and Piura river basins in Peru;
Flooding hazards and pandemic impacts within the Red River basin in Manitoba, Canada; and
Integration of Health and EO data and services for pandemic response in the greater New Orleans area in Louisiana, United States.
4.1. Previous Work
The OGC Innovation Program has included many initiatives over more than two decades addressing spatial data sharing for disaster management, emergency response, situation awareness, and other critical action scenarios. More recent relevant activities include the following.
Incident Management Information Sharing Pilot (IMIS) (2016) applied OGC principles and practices for collaborative development to existing standards and technology to prototype an IoT approach to sensor use for incident management and situation awareness.
Disasters Interoperability Concept Development Study (2018) focused on understanding how best to support the development or integration of SDI(s) for use in disasters.
Disasters Pilot 2019 (2019) focused on the demonstration of the usefulness of standards and SDI architecture within the Disaster community.
EO Cloud Platform CDS (2021) evaluated EO cloud platform architectures and alignment with open standards, as well as documentation of their readiness to support the Disaster Pilot 2021 disaster response exercise scenarios.
Health Spatial Data Infrastructure CDS (2021) drafted a common, standardized health geospatial data model and schema to establish a blueprint for better aligning and preparing the community for early warning, response to, and recovery from future health emergencies.
4.2. Analysis Ready Data
By some estimates, 80% of response time is spent procuring and preparing data, leaving only 20% to be spent learning from and acting on it. The more that this data preparation can be carried out in advance of data needs, the more that urgent time can be spent benefitting from the data itself. For this reason, Analysis Ready Data (ARD) is vital to support immediate responder decision-making. Many sources of data, however, particularly EO data, can be hard to find, complicated to share, difficult to access, and slow (or impossible) to process into common forms that are suitable for analysis and integration.
The most recognized definition of ARD is from the Committee on Earth Observation Satellites (CEOS): satellite data that have been processed to a minimum set of requirements and organized into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets…The definition of CEOS-ARD reflects the attributes of fundamental measurement products for the majority of global remote sensing users with land imaging applications, and are the minimum level required to support time series analysis and data interoperability.
CEOS have developed and are developing a number of ARDs Product Family Specifications for common EO types, such as Surface Reflection for optical imagery and Normalized Radar Backscatter for SAR imagery. There are a number of challenges to this specification process, besides the large and growing number of varieties of earth observation data including the following.
The type of analysis to be performed and/or the application for which the data is to be used have an impact on what is needed for data to be analysis ready.
The concept of readiness for whatever is the next step of analysis applies not only to “raw” earth observations, but also to any intermediate steps between collection and interpretation, including outputs from models such as predictive or classification algorithms, as well as other types of spatial data to be integrated, such as framework, reporting, or statistical data.
In Disaster Pilot 2021, a broader but less specific definition of ARD is developed, namely readiness or close to readiness of any dataset for broad categories of the next step in transforming and integrating that dataset for applications to improving disaster awareness. The concept of a geospatial data cube is useful here, in that datasets having a common definition of orthogonal space, time, and phenomenon axes are more likely able to be compared or combined for better coverage of a data space. An example would be combining EO data from multiple instruments for a longer time series or better cloud-free continuity. Another example could be combining observations or predictions made at different resolutions to provide a “drill-down” perspective. The point here is that the initial development of ARD concepts has proven very useful, but there is much more that could be done to reduce data “wrangling” for a broad range of datasets and analytical applications.
4.3. Clouds
Cloud computing is the capability to orchestrate remote computational workflows performed by essentially unlimited numbers of virtual processors, data stores, load balancers, and other processing components. Made possible by vast regional data centers containing hundreds of thousands of physical components and voluminous Internet bandwidth, cloud computing can reduce weeks or months of computational hardware acquisition and installation to a few minutes of laptop dashboard time. One of the few limiting factors in cloud computing capabilities is external: the cost, time, and difficulty of moving data into or out of a particular cloud computing environment or platform.
The cloud computing paradigm has a direct bearing on the use of EO data, satellite based or otherwise, for disaster awareness. The volume of Earth Observation (EO) has drastically increased in recent years, making it more likely that a particular disaster response can benefit from collected observations. An increasing number of satellites and improved capabilities (better spatial and temporal resolution) of new imaging sensors, as well as improved weather and other predictive models fed by them, have led to an exponential growth in volume, velocity, and variety of the daily EO data stream. EO data providers and users alike are faced with challenges in managing, processing, and handling this growth. Such volumes of EO data can no longer be downloaded and processed on local machines at either the necessary scales or speeds, nor for the variety of applications needed.
The cloud computing paradigm is able to meet these new requirements. As long as the raw observations can be efficiently uploaded to a cloud data center, new cloud-based EO exploitation platform data analysis workflows can then focus on bringing the much smaller scale application codes to where that data is stored. Cloud-based computing services can then provide virtual storage and computing resources that are deployed as needed and disposed of just as soon as they are no longer needed, so that overall IT costs for working with EO data can be significantly reduced.
As described in more detail in the EO Clouds Concept Development Study Report, this cloud computing applications-to-the-data paradigm is highly promising, and also highly disruptive to traditional concepts of data possession. As many of the participant component descriptions can attest, access to cloud computing can make all the difference in responding to urgent disaster awareness needs, but only if challenges can be overcome in producing and accessing analysis ready data, as well as delivering useful products from cloud workflows to people in disaster-impacted regions. Other challenges arise in adapting application codes to the specific cloud platform environment where input data are stored, or where multiple input datasets are stored on different platforms or regions.
The cloud is important but a great deal of technical, cultural, political, and educational work needs to be done to make it sufficient for improved disaster management based on earth observations.
4.4. Decision Ready Indicator Recipes
It has long been recognized, and is specifically called out in the Disaster Pilot 2021 CfP, that EO data need to be adapted — analyzed, integrated, and interpreted — in order to be of use in guiding the actions of disaster management stakeholders. As expressed in the CfP: “to bridge this divide, bringing together the Disaster SDI puzzle pieces that connect the right players from the data providers all the way to the first responders and the decision makers and everyone in between, forming a pattern that can adapt to any disaster, any region, any combination of data sources and tools.” This was seen as a way to “accelerate our ability to transform data from observation into decision…bridging the divides between providers, responders and other stakeholders, forming a connected ecosystem of data and technologies, and developing the capacity to produce Decision Ready Indicator (DRI) products that answer decision makers’ questions almost as fast as they can be posed.”
Through discussion and consultation between the Pilot participants and engaged stakeholders, a clearer concept has been developed for both the nature of DRI and the steps needed to generate DRI products from ARD sources.
There is a fundamental transformation between ARD that represents observations (physical, social, and computational) of geographic features in the world, and DRI that represents guidance on actions that stakeholders may choose and/or pursue.
The development of useful “recipes” for DRI products necessitates collaboration between multiple stakeholders as providers of ARD are unlikely to know well the particular decisions that need to be made, and decision makers are unlikely to know well how particular dataset can be used to guide them.
In the figure below are shown six representative user personas or roles for stakeholders in any ARD→DRI ecosystem. Responders and EO data providers are somewhat at each end of a DRI workflow. There are others, either in the middle such as analysts, or those ultimately affected by the result, such as the public. Each persona represents a unique set of responsibilities, knowledge, and interaction with a disaster information pipeline constituting an important role in devising DRI that are both feasible and effective.