2.1.  Terms and definitions

2.1.1. Cloud Computing

Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model is composed of five essential characteristics (on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service), three service models (Software as a Service, Platform as a Service, Infrastructure as a Service), and four deployment models (private, community, public, and hybrid cloud). Source: The NIST Definition of Cloud Computing.

2.1.2. Earth Observation

Data and information collected about our planet, whether atmospheric, oceanic or terrestrial. This includes space-based or remotely-sensed data, as well as ground-based or in situ data, and computer model based data such as weather forecasts. Based on https://earthobservations.org/geo_wwd.php (accessed on 2021-01-14), extended to include model-based data.

2.2.  Abbreviated terms

ADES

Application Development and Execution Service

AIS

Automatic Identification System

API

Application Programming Interface

ARD

Analysis Ready Data

AWS

Amazon Web Service

CDS

Concept Development Study

CEOS

Committee on Earth Observation Satellites

CFRA

Cloud Federation Reference Architecture

CMR

Common Metadata Repository

COG

Cloud-optimized GeoTIFF

CRR

Cross Region Replication

CWL

Common Workflow Language

DaaS

Data-as-a-Service

DCS

Data Centric Security

DIAS

Data and Information Access Services

DRM

Digital Rights Management

DWG

Domain Working Group

EC

European Commission

ECMWF

European Centre for Medium-Range Weather Forecasts

EMS

Execution Management Service

EO

Earth Observation

EODC

Earth Observation Data Centre

EODMS

EO Data Management System

EOSC

European Open Science Cloud

EOSDIS

Earth Observing System Data and Information System

ESA

European Space Agency

EU

European Union

EUMETSAT

European Organisation for the Exploitation of Meteorological Satellites

EWC

European Weather Cloud

GCMD

Global Change Master Directory

HDA

Harmonized Data Access

HPC

High Performance Computing

IaaS

Infrastructure-as-a-Service

IDN

International Data Network

JP2

JPEG 2000

MAAP

Multi-Mission Algorithm and Analysis Platform

NASA

National Aeronautics and Space Administration

NISAR

NASA-Indian Space Research Organisation Synthetic Aperture Radar

NIST

National Institute of Standards and Technology

NITF

National Imagery Transmission Format

ODIS

Ocean Data and Information Section

PaaS

Platform-as-a-Service

RCM

RADARSAT Constellation Mission

SaaS

Software-as-a-Service

SNAP

Sentinel Application Platform

STAC

Spatio Temporal Asset Catalog

STEP

Science Toolbox Exploitation Platform

SWOT

Surface Water Ocean Topography

TB

Terabyte

TEP

Thematic Exploitation Platform

UMM

Unified Metadata Model

USGS

United States Geological Survey

WPS

Web Processing Service

5.1.  Stakeholders

The Committee on Earth Observation Satellites (CEOS) identifies three key stakeholder groups of the EO data value chain in their ARD Strategy paper: (i) EO data providers (public and private), (ii) big data hosts and aggregators and (iii) data users. These stakeholder groups are mainly defined based on the value chain for EO data from satellites. For the purposes of this study, EO data encompasses in-situ as well as model-based EO data, in addition to satellite data. Furthermore, through the advent of cloud-based services and platforms for EO data management and analysis, the current EO system architecture landscape diversified and the three stakeholder groups defined by CEOS only partially reflect the EO data value chain. For this reason, rather than combining big data hosts and aggregators, we believe that the ‘aggregators’, how CEOS defines them, should be considered as data users. We therefore propose to differentiate between two groups of data users: intermediate users and end users. This differentiation is also reflected in the Copernicus Market Report 2019.

Subsequently, we describe the following four stakeholder groups: (i) EO data providers, (ii) cloud service / platform providers, (iii) Intermediate data users and (iv) end users.

5.2.  Evolution of EO System Architectures

The continued exponential growth of big EO data, combined with technological advancement, leads to fundamental changes in how and where EO data is stored and how users access, process, and use the data. The traditional ‘data-centric’ approach limits the full uptake and use of open EO data, as large volumes of EO data are copied to and processed on local machines (see Figure 1). In order to address the evolving need to minimize data duplication and offer scalable processing capabilities, the ‘moving code’ paradigm has evolved, which propagates the storage of large volumes of EO data as cloud objects and the access, processing and analysis of EO data on (cloud-based) servers. In general, cloud systems offer effective and scalable processing capabilities, but require advanced technical understanding from users. For this reason, more ‘user-centric’ approaches aiming to provide advanced access and processing of EO data, while hiding technical complexities, have been developed. Gomes et al. (2020) call these solutions ‘Platforms for big EO Data Management and Analysis’.
The current landscape of EO system architectures offers services of different evolution steps, from traditional EO system architectures to ‘user centric’ platforms to highly advanced cloud services allowing to execute entire applications on data stored in the cloud and federations of multiple cloud-based systems. Federations of cloud-based systems, however, is currently a visionary concept to which EO system architectures will evolve in the future. The role of the end user remains constant in all approaches and for this reason, we do not explicitly elaborate on end users.

5.2.1.  Traditional ‘Data-Centric’ approach

The majority of data users still tend to follow the traditional approach, where large volumes of data are downloaded and processed on local machines. In this scenario, different EO data providers (e.g. satellite-, model-based and in-situ data) are responsible for managing the data and disseminating it via a download service. Intermediate users are e.g. researchers or commercial companies, who make a copy of the data and pre-process and analyze the data on their local machines. With the growing volumes of EO data, this approach has gotten more and more cumbersome and limits the full uptake and use of EO data.

Figure 1 — Traditional data-centric approach, where large volumes of EO data are copied to and processed on local machines

5.2.2.  Traditional Approach With Cloud Storage

The growing volumes of EO data increase the need on the data provider side to more effectively manage EO data and offer data access in a programmatic way through e.g. an Application Programming Interface (API). Many EO data providers store their data archives on a cloud service (either their own cloud implementation or through services from commercial cloud vendors), which is accessible via an API for data users. The modality of how intermediate users access data does not change substantially in this scenario. Intermediate users still follow the traditional approach by ‘bulk-downloading’ large volumes of EO data to their local machines. The only change is the location where the data is stored, of which the intermediate user might or might not be aware of. Examples of this approach are the Copernicus Climate Data Store implemented by the European Centre for Medium-Range Weather Forecasts, as well as the public dataset programs Google Cloud Public Datasets and Earth on AWS.

Figure 2 — User-centric approach with dedicated platforms for EO data management and analysis

5.2.3.  Innovative ‘User Centric’ Approach — EO Data Management and Analysis Platforms

Platforms for EO data management and analysis have been developed based on the need to offer advanced access to and processing of EO data, while hiding technical complexities under a layer of abstraction. Different platforms and approaches have been developed. Examples for EO platforms are the Thematic Exploitation Platforms (TEP) developed by ESA, the Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform (MAAP) (Albinet et.al [1]), the Open Data Cube (Killough, 2018), as well as commercial platforms such as Google Earth Engine (Gorelick et al. 2018). An example from the climate community is the Climate Data Store Toolbox.

Figure 3 — User-centric approach with dedicated platforms for EO data management and analysis

These platforms represent a new ‘user centric’ approach, where users and applications are brought to the data in order to process and analyze it. Platforms differ in the underlying technology they use, level of openness, the level of abstraction, and the data and functionalities they offer. According to the charter of the OGC EO Exploitation Platform DWG, the platforms share a common set of functionalities:

• Cataloguing and searching;

• Storage and access;

• Visualization;

• Data processing and analysis; and

• User authentication, authorization, and accounting.

However, since current platforms for EO data management and analysis have been independently developed by public organizations and commercial companies, these platforms are not interoperable. In other words, they do not use (a) common (set of) interfaces and data formats for implementing the aforementioned functionalities. Additionally, the layer of abstraction dampens the flexibility and users might be constrained in the data or functionalities these platforms offer.

Regarding the EO data value chain, these platforms introduce platform providers as an additional stakeholder. Platform providers may be EO data providers, but most likely a different team in the organization or agency, or a third-party organization or company. The responsibility of managing, maintaining and provisioning the platform is shifted from the EO data provider to the platform provider. In the backend, such platforms may be operated in a cloud service, but may also be managed on a company server.

5.2.4.  OGC Activities Driving the Development of Interoperable EO Exploitation Platforms

The OGC has been driving the development of an interoperable Applications-to-the-Data EO platform architecture through multiple OGC Innovation Program initiatives (see Figure 4). Further initiatives — such as OGC Testbed-17 and the OGC Disasters Pilot 2021 — as well as the OGC EO Exploitation Platform Domain Working Group (DWG) will continue to improve and define the architecture in more detail.

Figure 4 — OGC Innovation Program initiatives that drove the development towards an interoperable Applications-to-the-Data EO platform architecture

At present, it is unclear which set of standards and specifications the OGC defines or recommends for realizing an interoperable EO platform. Defining this set of standards and specifications is amongst the key activities of the OGC EO Exploitation Platform DWG. The working group started writing an OGC Best Practice document for Earth Observation Application Package implementation (OGC 20-089). The document will focus on the application package (concept), but will also address the deployment viewpoint, i.e. how to deploy a package within a platform.

Relevant standards, specifications, and technologies appear to be, but are not limited to:

• Common Workflow Language (CWL) — To describe an application package (input and output parameters, invocation) as well as to define complex application workflows.

• Spatio Temporal Asset Catalog (STAC) — Used as data manifest for application inputs and outputs metadata.

• Docker — A docker container encapsulates the implementation of an application, and can be executed in different cloud environments. A docker container registry (also known as docker hub) provides a common means to store and download docker containers. A docker container can potentially also be built ad-hoc, in order to lower security concerns when downloading and executing pre-built third-party applications.

• OGC API — Processes and OGC Web Processing Service (WPS) — Provide a standard interface for deploying and executing an application. Deployment requires the transactional extension of the API / service. The interface is used by both EMS and ADES, which may require further profiling of the generic processing interface to fully specify the specific interactions and functions of the two services.

• OGC OpenSearch, with Geo & Time extensions, with EO extensions — For discovery and cataloguing.

5.2.5.  Cloud-Based Data Access and Processing

Compared to EO data management and analysis platforms, cloud-based systems represent a more flexible approach of the ‘moving-code paradigm’. They provide scalable and effective access to and processing of EO data in the cloud and eliminate the need to download massive amounts of data. While intermediate users are provided a higher degree of functionalities compared to EO data platforms, pre-defined data selections often remain. On the other hand, cloud-based systems demand deeper technical knowledge of the system configuration and this poses a challenge to many EO data users.
Many large data organizations are currently either moving their data holdings to commercial cloud providers (e.g. NASA), or are in the process of implementing their own cloud service (e.g. ECMWF). In this latter scenario, data providers turn into cloud service providers at the same time. Nevertheless, cloud-based services also differ in their functionalities and specialization. There are general cloud services from commercial providers, e.g. Amazon Web Services or Google Cloud Platform, offering high flexibility but demanding high technical knowledge from users. With the paradigm change in EO, more specialized cloud services tailored for EO access and management are being implemented, such as the Copernicus Data and Information Access Services (DIAS) or the European Weather Cloud implemented by ECMWF / EUMETSAT. They offer the same flexibility as more general cloud services in terms of VM specification, but offer facilitated access to EO data holdings as well as community specific tools facilitating EO data processing.
Instead of downloading EO data and processing it on local machines, intermediate users specify a virtual machine in the cloud and with this virtual machine, they access and process the data and only transfer intermediate or end results to their local machine. This represents a true paradigm change of how users access and process EO data. However, it also implicates challenges on different levels, including an insufficient expertise in cloud-based systems, a general skepticism in several aspects of cloud security and limited transparency in potentially evolving costs for processing. For these reasons, strong efforts have to be undertaken to strengthen capacities in cloud-based services in general, while building up overall trust in the security of cloud services and a good understanding for emerging processing costs.

Figure 5 — Cloud-based data access and processing

5.2.6.  Future: Federations and Interoperability of Cloud-Based Systems

A natural future evolution of EO cloud-based systems will be that interactions between these systems will increase, sharing data and processing in order to support the information needs of users. Thus, cloud federations will emerge, providing the ability to flexibly share data, applications and processing resources between multiple cloud-based systems.

Figure 6 — Federations of cloud-based systems

A number of challenges will have to be met in order to achieve federations of cloud-based systems, for example reaching a suitable level of interoperability and portability of cloud services and components, as well as being able to control and optimize costs when working within a federated system.

The European Open Science Cloud (EOSC) is an initiative that aims to build a federation of cloud services from both the public and the private sector, in order to enable the re-use of (research) data across borders and scientific disciplines. This is only possible if specific standards and guidelines are put in place for users and cloud-system providers. Users have to trust the data and services that are made available for re-use. For this reason, a central activity for EOSC is to define ‘rules of participation’ that enable users to assess the quality of the data and services offered in federated cloud systems (Bicarregui [2]). Another example of a federated system is WEkEO, which connects Copernicus data servers as well as processing services from different entities, for example the Earth Observation Data Centre (EODC)
for Water Resources Monitoring, which provides compute and storage resources as well as access to high performance computing facilities.

NOTE  The NIST Cloud Federation Reference Architecture (CFRA) identifies necessary and possible functions and capabilities to support general federations (of clouds), and organizes them into a reference architecture, without dictating the use and implementation of these functions and capabilities. Furthermore, the CFRA identifies a range of deployment and governance models. The CFRA can be helpful in getting a better understanding of the design of cloud federations in general, and understanding the nature of particular federated systems such as the EOSC.

6.1.  Data Coverage and Transmission to the Ground

When talking about big EO data, particularly with remote sensing data produced by satellites, it is useful to know the relative magnitude of the data, as well as which regions of the globe are typically covered, and how that data is actually delivered from satellites down to earth. This section covers these aspects.

Satellite data providers receive tens of Terabytes of new EO data every day (e.g. NASA EOSDIS: ~50TB, Maxar: ~80TB), and their EO data archives are in the Petabyte range (e.g. NASA EOSDIS: ~56PB, Maxar: ~120PB). The satellites can typically cover the whole globe, although commercial providers tend to focus more on inhabited regions of the world, and less on the poles.

The data is transmitted to ground stations, which are mostly self-owned, but in some cases also rented. It was reported that the time it takes to make new EO data available to users — including transmitting the data to the ground and processing it — usually takes a few days (e.g. for Maxar: less than 2 days). For certain high priority user groups, such as the military or in case of a disaster, that time may drastically be reduced (e.g. for Maxar: less than an hour). Emerging technologies such as in-orbit communications can help reduce the time it takes to deliver collected data to the ground — and thus the total delivery time — even further.

One cloud service provider reported that the antennas of rentable ground stations are directly located on top of their data centers. That means that the ground stations are situated close to the cloud environment. Bandwidth limitations for transmitting data — e.g. from the ground station owned by a satellite provider — into the cloud can thus be circumvented. Ground-based transmission would no longer be needed. Instead, the downlink would be directly into the cloud. In other cases, i.e. where there is sufficient bandwidth for transmitting data from the ground station into the cloud, HTTP based streaming directly into cloud object storage was mentioned as an adequate approach.

6.2.  Data Storage

As mentioned in the Introduction, vast amounts of EO data is being produced every day. Managing the growing flood of EO data within their own IT infrastructure is increasingly challenging for EO data providers. Cloud service providers offer a way out of this dilemma, since their cloud environments provide access to a whole network of server plants, with unprecedented levels of storage and processing capabilities. This section documents the results from meetings with both satellite data and cloud service providers on the topic of EO data storage, with specific focus on data storage.

As described in the previous section, satellite data providers nowadays manage up to hundreds of petabytes of EO data. It is increasingly challenging to support self-hosted IT infrastructure for such an amount of data. Satellite data providers therefore tend to move their data into the cloud. Planet Labs even started using the cloud for data storage right when the company was founded.

Maxar, for example, switched to the cloud in 2017, because their data center simply could not be expanded anymore, and an internal evaluation of cloud environments revealed that moving to the cloud would provide clear cost savings. Furthermore, the move enabled Maxar to put their focus back on satellite management, leaving the data center management to the cloud provider.

NASA’s Earth Observing System Data and Information System (EOSDIS) is migrating to the cloud as well. EOSDIS Distributed Active Archive Centers (DAACs), located throughout the United States, process, archive, document, and distribute data from NASA’s past and current EO satellites and field measurement programs. One by one, these centers are moving their operations into the cloud. A major reason for migrating to the cloud is the expected growth of data, due to new high data volume missions such as Surface Water Ocean Topography (SWOT) and NASA-Indian Space Research Organisation Synthetic Aperture Radar (NISAR). The annual growth of data is expected to be 50PB in 2022, with the archive size growing to be more than 246PB by 2025 — see figure X.

Figure 7 — Historic and projected EOSDIS archive size and annual growth (source: https://nasadaacs.eos.nasa.gov/eosdis/cloud-evolution)

Moving the EOSDIS data archive to the cloud is expected to improve management and accessibility of the data, and enables users to bring their applications (close) to the data, by executing these applications in the cloud. The aim is that users do not need to download data (e.g. satellite images) at all; instead the analysis is done directly in the cloud, and only ultimate results would be downloaded.

NRCan is also faced with a significant increase of data volume: while historic EO data currently amounts to ~3PB of data, the recently launched RADARSAT Constellation Mission (RCM) leads to an increased influx of data. The data archive size is expected to grow to ~10PB — exceeding the current storage capacity of the existing system (which is ~5PB). At NRCan, plans were therefore set in motion to migrate their EO Data Management System (EODMS) — an archiving and discovery system for the Government of Canada’s EO data — to cloud based storage (and processing). Within the OGC Earth Observation Applications Pilot project (conducted in 2020), an extension of EODMS was developed that runs in the cloud. Within the next 3 to 6 years, the entire EODMS architecture is planned to be migrated to the cloud. A major goal thereby is to reduce the need for users to download EO data and to encourage them to process the data directly in the cloud. This includes the production of value-added data, which is expected to be available faster with cloud-based processing. Reducing costs — for current data centric operations of EODMS — is another major reason for moving EODMS operations to the cloud. Last but not least, service uptime is expected to be better using the cloud, compared to current data center operations.

The migration of the historic data into the cloud can for example be done using mobile units with large storage quantities (up to Exabytes), for example the AWS Snowmobile. These units collect the data from the provider’s data centers and deliver it to one of the cloud provider’s data nodes, circumventing the alternative of uploading the data via an internet connection (which, for transferring 100PB of data would take ~3 months with an upload speed of 1000Mbit/s, and ~2.5 years with an upload speed of 100Mbit/s).

Cloud environments that provide services around the globe are typically structured into multiple regions. Data is transmitted into the cloud usually within a specific region. Processing that is performed on the data within that region is said to be done close to the data. However, when processing would be done in other regions, or data is requested from other regions, delays could occur due to the time it takes to transmit data between the two regions. Some cloud service providers therefore support replication of data across cloud regions. Cross Region Replication (CRR) allows data providers to replicate their data so that it is available as backup or for hot applications in multiple regions.

With respect to actual (meta-)data formats as well as open or proprietary software and standards for storing EO data, interviewees responded as follows:

• Many satellite data providers use Cloud Optimized GeoTIFF (COG) for storing satellite imagery data in the cloud. In addition, plain GeoTIFF, JPEG 2000 (JP2), National Imagery Transmission Format (NITF), Network Common Data Format Version 4 (netCDF4) were mentioned.

• Planet Labs primarily uses JPEG 2000 encoding when transmitting raw satellite imagery to the ground, and also for long-term storage of that data. Cloud Optimized GeoTIFF is produced on the fly, upon user request. Caching mechanisms (even user controlled, with a new system/service) are in place to reduce the amount of processing required to do the conversion from JP2 to COG. The preference for JP2 is due to the significantly smaller file size, compared to COG. It was reported that a 2-3 GB JP2 encoded satellite image would result in a ~70 GB COG.

• Maxar reported that they are developing standards to support their AI / ML applications. Metadata such as spatial resolution and other quality parameters are needed in order to advertise and determine the kind of imagery that these applications can use. When deemed useful for the wider community, this work is being brought to official standards organizations such as the OGC.

6.3.  Data Discovery

Since an enormous amount of historic EO data already exists, and more EO data is produced each day, being able to search for and discover relevant data is a critical function of any EO system.

Satellite data providers typically have some kind of portal that includes discovery functions. Maxar, for example, offers two options: A data catalogue website, where users can search (using a spatio-temporal bounding box), display and browse images (with resolution reduced to 15m), as well as an online platform for registered users (SecureWatch), which provides direct, on-demand access to all EO data in highest resolution. The Earth Observation Data Management System (EODMS) is NRCan’s portal to discover and download satellite data. ESA offers EO-CAT for browsing the metadata and images of EO data acquired by various satellites, as well as Earth Online — an EO information discovery platform. NASA has several discovery tools and services, such as Worldview, OPeNDAP servers, Earthdata Search, the Global Change Master Directory (GCMD), and the International Data Network (IDN) — the latter three of which use the Common Metadata Repository (CMR) as backend.

The CMR supports a range of metadata formats as input (e.g. ISO 19115), which are mapped to (and from) core metadata elements defined by the Unified Metadata Model (UMM), and a number of APIs and formats for access (e.g. OpenSearch, STAC, and CMR Rest APIs).

The Planet Explorer is Planet Lab’s GUI-based online tool for searching and analyzing geospatial imagers. In addition, Planet Lab supports a custom RESTful API for searching their complete imagery catalog. The API was built to handle over 1 billion metadata records. It was emphasized that such a number of records poses unique engineering challenges for building a performant, reliable, and scalable discovery system.

Metadata plays an important role for discovering relevant datasets. Interviewees emphasized that it is still difficult to discover new data sources for a given area. Catalogue formats and services could be of use, especially federations of catalogue services. The Spatio Temporal Asset Catalog (STAC) specification — a “common language to describe a range of geospatial information, so it can more easily be indexed and discovered” (source: https://stacspec.org/) — was often mentioned for discovery of satellite imagery within cloud object storage. Centralized access portals, such as the GEOSS Portal — with its discovery component, the GEO Discovery and Access Broker — are also helpful, as they aggregate metadata from multiple EO systems. Open source tools, such as GeoNetwork opensource and Magda, support setting up catalog applications. It would even be possible to set up a database in the cloud to index available data, and make the database publicly available. These approaches are all service driven, i.e. a user has a single entry point that is used to discover available data. A slightly different approach would be to have standardized web APIs that provide access to metadata about available resources (e.g. EO data), and let web crawlers harvest the information. OGC API — Records, which is currently in development at OGC, could be a candidate for such an API. Users could then query for relevant information directly in web search engines such as Google Search.

However, even if data sources are discovered based on matching spatio-temporal criteria, their metadata may not provide sufficient information needed for actually making use of the data. In order for applications to process some input data, that data needs to fulfill the requirements of the application. A satellite data provider emphasized that his AI /ML algorithms require detailed information before being able to use an unknown data source, such as information on the actual content (e.g. observed variables), which kind of processing has been applied, as well as data quality (spatial and spectral errors). While a multitude of metadata formats already exist today (e.g. ISO 19115, ISO 19157, ISO 19165, Spatio Temporal Asset Catalog (STAC)), potentially with a number of profiles and extensions, there does not appear to be a definite standard framework of definitions for the type of information the aforementioned algorithms require. This challenge is described in more detail in the section titled Lack of a Standard Framework for Specific Quality Factors for EO Data.

6.4.  Data Processing

Within the EO Cloud Platform architecture, EO data (pre-) processing is a key functionality required to support user applications. The raw data is typically processed multiple times, thus deriving higher-level information that enables decision making. This section documents the type of processing currently performed by satellite data providers, the software and tools used, and the processing challenges they are facing.

Satellite data providers typically pre-process the raw satellite imagery, for example doing orthorectification, band selection, and radiometric as well as atmospheric corrections. In rare cases, users do their own preprocessing to get better results (e.g. using better elevation models or ground control points). Multispectral data is typically requested by data scientists as well as value-added resellers and partners, while many users simply request RGB imagery. In specific cases, in-house analytics can also be done by experts from the satellite provider, on behalf of specific customers. For example, in a disaster situation, Maxar does forest fire detection using AI / ML techniques.

Some satellite data providers actively engage in the implementation of edge computing techniques. Edge computing allows the execution of pre-processing or even feature detection directly at the satellite before the data is transmitted to earth. This can help reduce the latency with which useful information is produced. It can also support the provisioning of actionable intelligence in disaster situations with low connectivity on the ground (for further details, see the section titled Delivering EO Data to Poorly Connected Regions).

During the interviews, satellite data providers were also asked if they allow users to upload (containerized) applications into their systems, to process their EO data. Some satellite data providers have tested this approach, and some users (e.g. the military) already make use of it. For example, Copernicus Data and Information Access Services (DIAS) offer processing using containerized applications close to the data provided by ESA’s Sentinel satellite missions. Docker was mentioned as an application container, for example encapsulating AI / ML applications.
However, some providers refrain from executing processing on behalf of users. They argue that users can build their own processing environment in the cloud, close to the data offered by the provider. Typically, that means using the same cloud (and region) in which the data is stored. However, one satellite provider mentioned that they can also bring the data to the user’s preferred cloud or processing environment. It is unclear which costs that approach would incur (for data transfer and storage in another environment) but apparently some users prefer to have this option.

One interviewee said that the likely reason why most users have not bought into the new applications-to-the-data paradigm yet is that so far intermediate users were responsible for providing value-added services. Others also noted that users may have high performance computing (HPC) facilities (potentially shared with other users), which they intend to continue using for EO data processing. In fact, one interviewee raised the concern that cost might significantly increase when shifting (their model processes) to cloud based processing, though a detailed analysis of actual costs would be needed to verify this concern. Nevertheless, most satellite data providers regard the new paradigm to be the future.

NASA EOSDIS is already actively engaged in moving EO data as well as processing to the cloud. For example, the Harmony project implemented data transformation services in the cloud (e.g. for subsetting, regridding, and reprojecting data) — allowing the production of analysis-ready data (ARD) on the fly for the user (Blumenfeld [3]). With data being available in ARD form, data can directly be processed in the cloud — rather than having to download the data first. NASA EOSDIS is using and supporting an ecosystem of tools that allow for high performance analysis in the cloud, in particular the software packages promoted by Pangeo — a community platform for Big Data geoscience — such as Xarray, Dask, and Jupyter. NASA EOSDIS has a strong commitment to capacity building and education, with the goal to enable data scientists to perform EO data analysis in the cloud.

Currently, the cloud environment of NASA EOSDIS does not allow users to upload their analytic applications directly into this environment. Instead, users need to build up their processing system in their own cloud accounts.

The following challenges related to processing of EO data were reported during the interviews:

• When satellite imagery is used to perform visual analysis, color changes from one image to another, due to atmospheric changes between the collection times, pose a challenge. Atmospheric compensation helps to a certain extent (especially for images taken by the same sensor), but a standard way of visually representing satellite images (for images taken by different sensors) was requested to be developed by the industry.

• Another challenging use case is automated analysis based on inference or information extraction. Well-defined quality metadata is needed in order to provide useful results. Taking information about spatial errors as an example. Every satellite has its own spatial error. Small satellites can potentially be off by 100 meters in absolute position. How to do co-registration (combination) of data from multiple satellites if the spatial errors are not well defined? The spatial errors of the resulting product would be undefined. However, the precision of absolute positions is relevant for disaster response, for example to route resources to a certain location. With well-defined information on spatial errors in input data, algorithms that derive new information from that data should be able to create an estimated error of the precision of a computed location. The resulting error value would be useful for decision makers. For example, an error of 100 meters may not be so critical in flat, open land, but could make a huge difference in a metropolitan area — especially in a disaster scenario. In that case, first responders need to have the means to understand how accurate — and thus trustworthy — some data is. Decision making based on EO data often leverages higher-level information derived from other EO data by some kind of process (including multiple processes). In order to support auditing requirements, the lineage / provenance of derived data must be available (and kept for a certain period of time, e.g. 10 years). If sufficient metadata is kept, derived data can even be reproduced. Keeping sufficient metadata records to support auditing is an important operational — and challenging — aspect for any processing system that produces higher-level information for decision makers.

6.5.  Data Access

With EO data having been produced, processed, and stored by the provider, and discovered by the user, we will now summarize how users are currently able to access EO data.

6.6.  Security

Within IT systems in general, and EO systems in particular, security is an important — and cross-cutting — concern. When data is transmitted, in use, or at rest, a real-world system must ensure that the data is secure. The goal is to ensure data confidentiality as well as integrity and availability of the data and the system itself. Cloud service providers specialize in providing customers with all means necessary to achieve this goal. The interviews revealed that satellite data providers which host their data in the cloud have great confidence in cloud environments being secure.

Here are some noteworthy results from the interviews on the topic of security:

• Once the data leaves the system, security can no longer be guaranteed, as one interviewee reported with specific regard to data integrity. Data integrity concepts typically stop at the edge of the system boundary.

• A satellite data provider remarked that he was required by law to keep all satellite data that is collected. The cloud environment provides multiple backups of data, in separate physical locations. The satellite provider further noted that whenever new data is being derived from other datasets (e.g. the original satellite data), then extensive metadata and provenance information is being kept — which allows to re-create the derived product if necessary. Taken together, that leads to the satellite data provider considering the risk of losing data when hosting it in the cloud to be very low.

Many times, OGC Innovation Program initiatives have analyzed, developed, and tested security mechanisms and technologies. OGC Testbed-16 investigated federated security as well as data centric security. The OGC Testbed-16 Federated Security Engineering Report (OGC 20-027) [9] analyzes aspects of security and trust in a federated computing environment, as defined in the NIST Cloud Federation Reference Architecture. Possible approaches covered by the report range from traditional methods for securing basic communications between federated entities, to the use of emerging security technologies such as trust frameworks, blockchain, and zero trust architectures. The OGC Testbed-16 Data Centric Security Engineering Report (OGC 20-021r1) [8], on the other hand, is specifically concerned with securing data itself, ensuring data confidentiality and integrity even in case that data leaves the system of the data provider. Both reports are relevant for securing cloud-based systems.

Data Centric Security (DCS) is about encrypting the data itself whenever it may be stored outside of a fully trusted environment, and controlling access to certain parts of the data, using role-based authentication and access. Examples of storage without full trust can be cloud environments provided by an external service provider, as well as mobile devices. One scenario investigated in Testbed-16 assumes that a user operates in an offline mode, disconnected from the network — which is an important assumption for disaster response operations. Protected data required for a “mission” is stored (in encrypted form) on a mobile device for later field use. DCS mechanisms ensure that access to the data will expire at some point in time in the future (suitable for mission requirements), and that access to the data is limited based on user roles (e.g. the mission commander may have access to more sensitive information).

National and international laws can have an impact on data confidentiality. National laws may force a company, such as a cloud service provider, with business dependency in that nation to disclose customer data (e.g. because the customer broke the law, or for reasons of national security) — potentially even if the data is stored on servers outside of that nation. Such laws may be in conflict with laws of other countries, creating an unclear situation for everyone involved (especially cloud service providers and data owners). Celeste and Fabbrini [6] describe this issue in more detail. It is unclear if data centric security can help solve this issue, without compromising the key concepts behind the Applications-to-the-Data paradigm, and its realization within the EO Cloud Platform architecture.

The applications-to-the-data paradigm also raises security concerns regarding user applications that are executed within the confines of a data provider. This issue has been investigated to a certain extent within the OGC Earth Observation Applications Pilot (for further details, see section 5.8 of OGC 20-073 [7].

7.1.  Towards the Production of Higher Value, “Living” Data

Satellite data providers intend to increase the number of satellites. Areas of the earth will be covered with a higher frequency, with different sensors that produce different types of information, which can help to fill important gaps within currently available data.

The amount of generated EO data will increase exponentially. Cloud environments are expected to support handling the increased influx of data. In addition to increasing the overall coverage, satellite data providers also plan to decrease the latency with which data becomes available for decision making (e.g. at Maxar from ~60 minutes down to tens of minutes). AI / ML techniques as well as Edge Computing and the New Space paradigm were mentioned as enabling technologies.

Satellite data providers will thus be able to shift from primarily supporting mapping projects to supporting both mapping of the earth, detecting relevant changes on the earth, as well as providing inputs for computational models — and that shift has already started. Users will be able to work with living datasets. For example, rather than working with building data, i.e. the state of a building as currently known, a user will be able to work with changing building data — subscribing to data updates that will frequently be provided (with the help of satellite imagery and change detection algorithms). Similarly, data scientists will be able to make use of high-volume, n-dimensional data, for example from NASA satellites.

7.2.  Open vs. Proprietary

Unsurprisingly, satellite data providers use both open and proprietary standards and tools in their system — and intend to continue doing so in the future. Since customers are requesting adherence to open architectures, and since open frameworks as well as open source tools, systems, and platforms — e.g. deep learning frameworks for AI/ML applications (such as TensorFlow), and Kubernetes for orchestration of containerized applications (supporting automation of application deployment, scaling, and management) — have become available and are actively being developed and improved, “open” is being embraced more and more.

7.3.  Improving Interoperability

Many satellite data providers are members of global disaster programs, such as the International Charter — Space and Major Disasters. Surprisingly, these programs do not appear to define specific technical requirements regarding formats and interfaces for accessing EO data during a disaster response, but rather provide a network to bring disaster response and data providers together.
In case of a disaster, data requests are sent out to program members. The members then typically make data of the disaster area available, beginning with the latest data before the disaster event, followed by new data generated during the program activation. Maxar, for example, publishes the data on their open data portal, which supports subscriptions to new data, allowing relevant disaster response stakeholders to be notified as soon as new data has been published on the portal.
While EO data is being made available for use within a disaster response, the data provider chooses the data formats as well as the interfaces via which the data is published. This may complicate the integration of data from multiple sources. A common framework of standards and specifications for accessing EO data would help to improve the situation. The OGC can play an important role in developing such a framework.

More and more, satellite data providers strive to integrate data from multiple sources themselves, in order to derive new and valuable information of interest to users. Maxar, for example, is pushing the implementation of AI/ML-based multi-sensor, multi-intelligence analytics. For example, they use RADARSAT-2 (from another data provider) if the cloud cover above an area is too high to use their own satellites for detection of fires or burnt areas. Another use case involves tapping into Automatic Identification System (AIS) signals emitted by ships, and combining this information with results of ship detection algorithms that are run on satellite imagery, in order to identify “dark ships”, i.e. ships that have not activated AIS. It was noted that such integration scenarios also require a common framework of standards and specifications. Specific emphasis was put on the current lack of a framework for spatial and spectral error propagation (further details are given in [ref OGC 20-088]), as well as certain metrics that would support AI applications (use case: given some image metadata, it should be possible to tell which objects can be extracted from the image).

To summarize: There is a need for defining a framework of standards for encoding relevant EO data and metadata, as well as interfaces and formats for efficient and interoperable access to the data. Modern analytics on EO data — such as feature detection — require more detailed information about given data sources in order to work well and to create information that is useful for decision makers.

7.4.  Continued Migration to the Cloud

Some satellite data providers have only started to move their EO data and relevant operations into the cloud. These providers typically want to gather experience with the new (cloud) environment before fully committing to it, and need to avoid disruptions of their everyday operations. Consequently, plans are being made for the continued migration of their data into the cloud.

NASA EOSDIS, for example, started by moving the operations of NASA’s Global Hydrology Resource Center Distributed Active Archive Center (GHRC DAAC) into the cloud. Other DAACs will follow in the coming years (source: https://nasadaacs.eos.nasa.gov/eosdis/cloud-evolution).

NRCan also started moving to the cloud, prototyping an extension of their EO Data Management System (EODMS) that runs in a cloud environment. The migration of EODMS to the cloud is planned to be completed within the next 3 to 6 years.

EODMS users should have a similar experience when accessing data provided by EODMS as they have nowadays. However, the processing capabilities would improve drastically. While currently there are bottlenecks regarding the amount of data that can be stored and processed, processing in the cloud, close to the data, is expected to provide all the processing power that is needed. Whether EODMS users fully shift to processing EO data in the cloud (e.g. for executing common pre-processing tasks), or whether they keep using their already existing high performance processing infrastructure — only expanding into the cloud when the situation demands it — remains to be seen. Costs need to be considered — of local and cloud-based storage and processing, as well as a hybrid of the two. The topic of costs is further discussed in the section titled Controlling Cloud Costs.

A similar situation presents itself at the department of Fisheries and Oceans Canada. A number of EO user groups still download EO data, for example from multiple space agency sites (such as ESA, NASA, NOAA, CNES, JAXA, USGS Earthexplorer, and EUMETSAT) and process it locally. Other groups have started to explore cloud computing. The Ocean Data and Information Section (ODIS) already migrated their processing to the cloud. While some user groups are reluctant to change their current workflows (involving local storage and processing), they acknowledge that large data transfers are an issue — which is one of the main arguments for adopting the applications to the data paradigm. The intention of ODIS is to support other groups fully migrating to the cloud as well. That is in line with Canada’s cloud-first strategy (for further details, see the Government of Canada White Paper: Data Sovereignty and Public Cloud).

7.5.  Expanding the Toolsets

A number of software tools have already been developed that allow for high performance analysis of EO data in the cloud. The software stack promoted by Pangeo is one example, the Sentinel Application Platform (SNAP) — part of the toolboxes on ESA’s Science Toolbox Exploitation Platform (STEP) — is another. Satellite data providers such as ESA, and platform providers such as WEkEO, are going to continue working with the EO user community to develop new and improve / evolve existing tools, toolboxes, and platforms, making it easier for users to work with the ever-growing amount of EO data.

8.1.  Lack of a Standard Framework for Quality Factors for EO Data

Applications that perform analytics on satellite imagery (e.g. feature extraction) need a specific level of image quality. Quality factors include, but are not limited to: information on spatial and spectral errors (e.g. atmospheric correction and geolocation accuracy). A standard framework needs to be developed that provides clear definitions for these quality measures, to be included in image metadata ([ref OGC 20-088] provides further details). With such definitions contained in image metadata, applications can evaluate if the image data fits the needs of these applications.

The ability of an application to determine whether a given data source can be used or whether the data does not fulfill the minimal requirements is critical in disaster response scenarios. A disaster response manager may find multiple applications that would be useful to his work in a given crisis, but currently it is difficult to evaluate — fast and ideally in an automated fashion — which applications can actually be used with available datasets.

A particular EO Cloud platform may provide certain guarantees regarding the readiness of satellite data for analytics (supporting certain levels of preprocessing, quality information, and metadata to be available) which an application developed for that platform may rely upon. However, developing an application making use of data from multiple platforms is impeded due to the current lack of interoperability between EO Cloud platforms and a framework of standards regarding applications, services, and (meta-) data. OGC working groups such as the OGC EO Exploitation Platform DWG as well as the OGC Data Quality DWG would be good places to pursue the necessary standardization work. In fact, the charter of the OGC EO Exploitation Platform DWG mentions the definition of a standard set of metadata.

8.2.  Cloud Interoperability

Ideally, EO Cloud Platform implementations support multiple cloud service environments. Using components that support open standards — both formats and APIs — so that the underlying cloud specifics can be hidden, would facilitate this goal, and lead to an interoperable system architecture.

Satellite data providers that moved their data into the cloud typically prefer working with a specific cloud provider. However, some of them also support other cloud environments (e.g. due to customer demand), and thus support a multi-cloud approach. For these providers, cloud vendor lock-in was not considered to be an issue. Others noted that cloud interoperability and portability is an important topic. However, introducing levels of abstraction into the architecture to hide cloud specifics comes at the cost of not being able to profit from the benefits that cloud native services, for example, offer. At least for satellite imagery it was noted that cloud object storage, with HTTP based access to files using standard formats (such as [cloud-optimized] GeoTIFF), could quickly be ported to a different cloud service provider. In general, the use of open standards was considered to be helpful regarding system portability. Furthermore, by building a cloud-based system with small components (keyword: microservices), the expectation is that the portability of the system will increase.

The needs of EO users should also be taken into account. Interoperability is seen as a key requirement. A federated system with different access APIs and data formats is difficult to handle for the common user. Therefore, the provision of higher-level information products, created by processing EO data (potentially using complex workflows), should be achieved using open standards — both formats and interfaces / APIs. A uniform access layer still seems to be a highly desirable achievement, also for EO systems based on cloud technology. Cloud-native services as well as custom APIs and tools could still be used for analyzing the data and for creating higher-level information products. A set of best practices should be developed to define the standards and specifications with which interoperability between cloud-based systems can be achieved.

8.3.  Lack of Policies for Data Sharing in Case of Disasters

Bureaucracy was mentioned as a challenge for sharing data during a disaster. Apparently, there is a lack of clear policies defining which information can be shared with which stakeholders, and how quickly the information can be shared. For example, Maxar noted that in one wildfire disaster scenario, information about burnt houses was only allowed to be shared 7 to 10 days after the houses burnt down. In another disaster scenario, data provided by Maxar was not used because data rights as well as sharing policies were not clear to disaster managers. NRCan reported that managing a plethora of data licenses is challenging when trying to determine which users can access what type of data.

Policy makers — on national and international level — need to make sure that data sharing in case of disaster is facilitated by putting relevant policies in place. These policies should address questions of which data may be shared with whom and when in case of a disaster. They should also address questions of liability. Data licensing will play an important role as well. Many EO data sets are already made available under an open license, which is beneficial for sharing data during a disaster response. The goal should be to have well-defined, ideally standardized rules and definitions for data sharing, which can be applied in case of a disaster. This could go as far as enabling automatic evaluation of these policies by EO systems, allowing stakeholders to quickly identify which data they may share and use.

Digital Rights Management (DRM) and Data Centric Security (DCS) may play a role in solving this challenge; emerging projects such as iSHARE may provide valuable concepts and mechanisms as well. Necessary definitions — for example for data products such as detected objects — need to be developed. Semantic technologies may play a role as well, for evaluating the equivalence of definitions developed by different entities. Having an international standard of relevant definitions would be ideal, and — once developed — could be mandated by policy makers. The members of the International Charter — Space and Major Disasters could also play a vital role in the development of policies, best practices, and standards for data sharing.

8.4.  Delivering EO Data to Poorly Connected Regions

A major challenge in case of a disaster in poorly connected regions of the world is getting larger amounts of data to emergency response headquarters and personnel in the field. One satellite provider explained that in such a situation, their satellite imagery would currently be provided by sending it on hard drives.

In the near future, edge computing could help bring information from EO data to poorly connected regions: Analytics on EO data can be applied on the edge (e.g. in space), extracting relevant information (e.g. detected features) from big EO data and only transmitting the distilled results, to be used for decision making. Furthermore, some cloud service providers have a service for moving data — and processing power — from the cloud to a user using edge computing devices (see for example the AWS Snow Family). However, that process may take a considerable amount of time; it was reported that it could take several days, depending on the amount of data and the delivery destination.

One of the published OGC standards is of particular interest for addressing the challenge of getting data to poorly connected regions: GeoPackage.

QUOTE: “A GeoPackage is an open, standards-based, platform-independent, portable, self-describing, compact format for transferring geospatial information. The GeoPackage standard describes a set of conventions for storing the following within a SQLite database: vector features, tile matrix sets of imagery and raster maps at various scales, extensions.” (source: https://www.ogc.org/standards/geopackage)

A GeoPackage represents a file-based container for geospatial data. GeoPackages can be loaded onto mobile devices for offline use, with subsequent synchronization by GIS software when the user is back from the field. OGC Innovation Program initiatives drive the development of the GeoPackage standard, the latest installment being OGC Testbed-16 (see OGC 20-019 [10] for further information). The OGC Disasters Pilot 2021 would be a good opportunity to test the delivery of satellite imagery and derived products such as detected features within a GeoPackage to poorly connected regions.

8.5.  Cloud Optimized Data Formats

Satellite image data has always been big data. New satellite missions will result in a significant increase of data volumes. For example, the NASA-ISRO SAR Mission (NISAR) — to be launched in 2022 — is going to produce roughly 85TB of data per day (source: Blumenfeld [3]). Storing, processing, and disseminating such large amounts of data is a challenge with current technologies. Cloud service providers therefore actively engage with research and user communities to develop efficient mechanisms and data formats that allow constant storage as well as constant streaming of data. Cloud-optimized GeoTIFF (COG) is an example of a data format that was specifically designed for cloud environments. However, this format requires a lot of storage space, compared to other formats such as JPEG 2000. Developing optimizations that support storage, processing, and dissemination of the next generation of big data will be a major challenge.

Signell (2020) describes an approach that allows efficient reading of files in the cloud, particularly for formats that are not cloud-optimized. It was noted that the approach could be improved by:

• adding common compression/filtering schemes used by NetCDF4, HDF5 and GRIB2 to numcodecs, and

• developing a binary representation of the fileReferenceSystem metadata (replacing the use of JSON, which is inefficient for very large datasets such as millions of chunks).

8.6.  Controlling Cloud Costs

Cloud use incurs some level of cost. A satellite data provider can typically define the cost related to data storage, and control the cost for processing performed by the provider himself. However, transfer of data out of the cloud or to external services within the cloud (in the same or a different region) is hard to control if the number of users is unknown. Cloud service providers may have different cost models for the aforementioned transfer cases. Transfer within the same cloud (and region) usually has a lower or no cost at all than transferring data out of the cloud. That is a reason why processing should be performed within the cloud, close to the data. Still, what to do if data is publicly accessible and theoretically anyone in the world can download the data? AWS has the concept of Requester Pays for Amazon S3 data stores. That would move the transfer cost to the user. However, that may not be an option for an organization such as NASA, where policies (e.g. the NASA open data, services and software policies) are in place which require that users can download data for free. The transfer cost would then be the burden of the data provider. Organizations need to ensure that the total cost of their cloud based system is within their budget. They may even be legally liable if the costs get higher as the budget allows (see for example the U.S. Antideficiency Act). Cloud use can therefore be a financial challenge.

NASA EOSDIS modelled data egress, i.e. transfer of data out of the cloud, based upon decades-long experience with data egress from their Distributed Active Archive Centers (DAACs), experience gained so far with cloud data storage and use, as well as studies on expected data egress for future missions, for example the NASA-Indian Space Research Organisation Synthetic Aperture Radar (NISAR) (Cassidy [5]). The model allows transfer costs to be calculated, and ultimately to calculate the total cost for the cloud use. Still, actual data egress could exceed expectations, which is why EOSDIS developed and deployed a software implemented data flow controller, which ensures that data download traffic is within allowed bounds at any given time.

Avoiding data transfer as much as possible to reduce costs results in increased processing in the cloud. While that is typically seen as beneficial, it also comes with a cost. These costs need to be understood and carefully weighed against each other. In addition, processing in one cloud may be cheaper than doing so in another. The cost of transferring data between two clouds once, and then processing the data multiple times, may ultimately be less than doing everything within a single cloud.

Cloud cost optimization is an important topic. Several companies already offer related services. When performing an analysis of EO data in the cloud, being able to estimate the costs that the analysis will incur is critical to know — to avoid any overspending. Identifying the costs of executing a certain process in the cloud in advance is an open challenge.

8.7.  Capacity Building

Bringing processes to the data within the cloud is a paradigm change for analyzing big EO data. However, in order to create and execute processes in the cloud, users — but also providers — need to develop new skills. That can be a challenge.

NASA EOSDIS, for example, addresses this challenge in multiple ways. On the one hand, they develop, provide and extend learning material (webinars, tutorials, workshops, and articles). On the other hand, they establish focus groups to train different user groups and gather feedback from them. Finally, other community efforts are being leveraged and actively supported — especially Pangeo.

Likewise, NRCan plans to encourage their users to process data directly in the cloud, once the full migration actually starts. The idea is to follow an open approach, having a consultation process for users, as well as engagement with industry. Universities may also help in capacity building, by teaching relevant skills.

At Fisheries and Oceans Canada, Ocean Data and Information Section (ODIS), the aim is to gently encourage and actually help users to develop the necessary skills for working in the cloud, through piloting and testing activities that users should engage in. Demonstrating the values of moving to the cloud — such as cutting out the time and cost associated with download of large EO datasets — is seen as a good incentive.

WEkEO service providers are actively engaged in supporting users, both for urgent questions and for developing required skills, through: a helpdesk, publicly available material (e.g. user guides, webinars, and tutorials), as well as outreach and training (e.g. demonstrations, workshops, and hackathons).

8.8.  Working in Multiple Clouds

When EO data is stored and processed within the same cloud (and region), the implementation of the applications-to-the-data paradigm is relatively straightforward. However, when storage and processing occurs in different clouds and/or cloud regions, data essentially has to be transferred from place to place — which causes delays and costs. Identifying the most efficient way for performing analysis in such an environment — considering the variables cost and processing time — is a challenge.

8.9.  Developing Standards That Support Scalability of Applications for Big EO Data

Satellites produce tens to hundreds of terabytes of EO data each day, with the amount of data expected to increase significantly over the next 5 years — with the trend expected to continue going up in the next decades. This tremendous amount of data poses great engineering challenges.

One satellite data provider emphasized that in order to support the development of scalable systems, standards organizations should focus on standardizing small, reusable system components ((meta-) data formats, APIs, etc). These standards should be flexible / extensible to cater for unexpected needs — which may subsequently lead to standardized extensions. Rather than standardizing whole architectures, best practices should be developed.