Publication Date: 2020-10-22

Approval Date: 2020-09-23

Submission Date: 2020-08-25

Reference number of this document: OGC 20-034

Reference URL for this document:

Category: OGC Public Engineering Report

Editor: Christophe Noël

Title: OGC Earth Observation Applications Pilot: Spacebel Engineering Report

OGC Public Engineering Report


Copyright © 2020 Open Geospatial Consortium. To obtain additional rights of use, visit


This document is not an OGC Standard. This document is an OGC Public Engineering Report created as a deliverable in an OGC Interoperability Initiative and is not an official position of the OGC membership. It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, any OGC Public Engineering Report should not be referenced as required or mandatory technology in procurements. However, the discussions in this document could very well lead to the definition of an OGC Standard.


Permission is hereby granted by the Open Geospatial Consortium, ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.


This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.

Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications.

This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.

Table of Contents

1. Subject

This Engineering Report (ER) describes the achievements of Spacebel as a Platform Provider in the OGC Earth Observation Applications (EO Apps) Pilot and the lessons learned from the project.

2. Executive Summary

2.1. Objectives

The objective of the project is to build a “real world” environment of the Earth Observation (EO) Exploitation Platform architecture which has been developed over the last two years as part of various OGC Innovation Program initiatives.

The first phase of the project invited Application Developers to join a requirements definition workshop: developers were informed of platform capabilities, then asked to express their requirements in terms of data discovery, data loading, data processing and result delivery. The definition of use cases directed the implementation and evaluation of the second phase: EO platform operators (including Spacebel) implemented the necessary user infrastructure to enable application developers to achieve the goals of their use cases.

As stated in the Call for Participation (CFP) of the EO Apps Pilot, the Testbed-13 reports are considered relevant here because they help to understand the design decisions in context, but the documents are superseded by Testbed-14 reports which describe the target architecture.

The architecture is essentially based on two main components:

  • Execution Management Service (EMS): provides a RESTful interface (defined using OpenAPI) to register applications and build workflows from registered applications. The EMS selects the appropriate Application Deployment and Execution Service (ADES) platform to execute the processes based on the runtime input parameters (close to the data).

  • Application Deployment and Execution Platform: allows to deploy, discover, and execute applications or to perform quoting requests.

2.2. Proposed EO Platform

As a Platform Provider, Spacebel proposed the Walloon EO Regions! Platform based on the specified "Geospatial Exploitation Platform" implementation. The platform’s different components and containerized applicative service chains (Docker) are running on a Kubernetes cluster on a public cloud infrastructure (Google Cloud Platform).

Application Developers selected by OGC are able to deploy their applications via application packages on the platform and execute the applications through the required specified interfaces (EMS, ADES etc.). Spacebel would support these application developers to achieve this result.

The EO Regions! Platform was originally funded by the Walloon region and is being operated by Spacebel. EO Regions! is the Walloon hub of the European EUGENIUS network and is recognized as a Copernicus Relay for Wallonia by the European Commission for the promotion and support to users of Copernicus data. The main user facing Web Portal is available at and provides more background information about the platform and its objectives.

2.3. Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:


Name Organization Role

Christophe Noël

Spacebel s.a.


2.4. Foreword

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

3. References

4. Terms and definitions

For the purposes of this report, the definitions specified in Clause 4 of the OWS Common Implementation Standard OGC 06-121r9 shall apply. In addition, the following terms and definitions apply.

● Container

A standardized unit of software (Docker).

● OpenAPI Document

A document (or set of documents) that defines or describes an API. An OpenAPI definition uses and conforms to the OpenAPI Specification (OpenAPI)

● OpenSearch

Draft specification for web search syndication, originating from Amazon’s A9 project and given a corresponding interface binding by the OASIS Search Web Services working group.

● Service interface

Shared boundary between an automated system or human being and another automated system or human being

● Workflow

Automation of a process, in whole or part, during which electronic documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules (source ISO 12651-2:2014)

4.1. Abbreviated terms

  • ADES Application Deployment and Execution Service

  • AOI Area Of Interest

  • AP Application Package

  • BPEL Business Process Execution Language

  • CFP Call For Participation

  • CWL Common Workflow Language

  • DWG Domain Working Group

  • EMS Execution Management Service

  • EO Earth Observation

  • EP Exploitation Platform

  • ER Engineering Report

  • ESA European Space Agency

  • GUI Graphical User Interface

  • JSON JavaScript Object Notation

  • MEP Mission Exploitation Platform

  • OWC OWS Context

  • REST REpresentational State Transfer

  • TEP Thematic Exploitation Platform

  • TIE Technology Integration Experiments

  • TOI Time Of Interest

  • UI User Interface

  • URI Uniform Resource Identifier

  • URL Uniform Resource Locator

  • VM Virtual Machine

  • WKT Well-Known Text

  • WCS Web Coverage Service

  • WFS Web Feature Service

  • WPS Web Processing Service

  • WPS-T Transactional Web Processing Service

5. Overview

Section 6 introduces the project context, the initial requirements and baseline.

Section 7 discusses the Earth Observation Exploitation Platform aspects.

Section 8 presents the lessons learned during the project as a Platform Provider.

Section 9 describes the Technology Integration Experiments (TIE) achieved by Spacebel and Application Providers.

Section 10 provides the conclusions and recommendations for future work.

6. Initial Context

The section provides an overview of the project requirements and the tentative baseline agreed between the Platform and Application providers.

6.1. Initial requirements

The CFP detailed a set of evaluation criteria for ranking the proposed platform of the applicants. Additionally, some starting points for improving the architecture were also identified. An overview of the main topics to be explored during the pilot is provided below:

  • Discovery of applications, of input EO data and analysis results.

  • Provision of data, typically using the OGC Web Coverage Service (WCS) standard as an agnostic interface to data.

  • ADES and Application Package (AP) Constraints.

  • Workflow language, in particular reevaluating the Common Workflow Language (CWL).

  • Multiple outputs.

  • Trust, Deployment and Security.

  • Developer support in particular through the access to the execution logs.

  • Billing and Quoting Model

  • Communication Message Exchange (JSON rather than XML).

  • Asynchronous Communication such as a web hook callback.

  • Execution and Deployment on a (Kubernetes) cluster.

Participants shall not explore all these topics during the pilot, but experiences matching some aspects shall be captured as reflected in this engineering report.

6.2. Tentative Baseline

During the Kick Off meeting, participants & staff tried to find an agreement for a common baseline of features supported by the various platform.

As no presentation really covered a detailed description of the EMS and ADES components, those aspects have been unfortunately agreed with a very limited knowledge of the actual architecture.

6.2.1. Data Discovery

Data Discovery can be performed through either OpenSearch (two-steps) queries or OGC WMS/WCS API.

Despite that the data discovery aspects are not yet properly covered in the Testbed architecture, Testbed-14 already introduced the OpenSearch API for selecting the EO data inputs submitted to the process.

The Spacebel platform already includes an OpenSearch Catalogue, and detailed product metadata can be requested in the OGC O&M EO profile format (OGC 10-157r4). The Web Map Service (WMS) and WCS are not supported.

6.2.2. Data Retrieval

Data can be retrieved by an internal stage-in (performed by the ADES), through WMS/WCS, or using Object Storage.

The Spacebel platform stage-in requires mounting local data resources to the Kubernetes cluster node. In addition to the nominal stage-in, it is not perfectly clear if the proposed alternatives offer direct (network) access from the application, or if they suggest the support of those protocols for provisioning local files (as for the stage-in).

The Spacebel platform fetches all remote HTTP and FTP URLs and makes them available through the local file system to the processes. As discussed during OGC Testbed-14, the application descriptor should typically describe if the process supports natively HTTP or Object Storage access.

6.2.3. User Interface (EMS/ADES)

The agreed ADES/EMS interface is based on the draft OGC API – Processes specification, with REST JSON bindings, and Web Processing Service (WPS) standard.

The Spacebel platform required to be aligned with the work performed during OGC Testbed-14. However, the OGC repository hosting the mentioned OpenAPI specification does not exist anymore, so Spacebel shared its own version (OGC-ADES-Hackaton) reviewed during the 2019 OGC API Hackathon that was held in London [1].

The baseline also recommended to explore the handling of asynchronous callback. Because most application providers focus on the platform’s user interface, the priority was not given to this aspect during the project.

6.2.4. Workflows

The agreed Application Package is the CWL profile defined in Testbed-14. Additionally, bulk (parallel) processing might be investigated.

Based on the Application Providers feedback, Spacebel essentially worked on the parallelization approach, as described further in the document.

6.3. Tentative Grouping

During the Kick Off meeting, Spacebel discussed with Application Providers and tried to identify potential candidates to collaborate with. The following participants agreed to work on the EORegions! platform:

  • Computer Research Institute of Montreal (CRIM) acting as an Application Provider, but also reproducing a multi-platform workflow execution as demonstrated during the OGC Testbed-14 project

  • European Union Satellite Centre (SatCen) with the Coherence application.

  • Pixalytics with the Mosaicing application.

  • German Aerospace Center (DLR) with the Urban Indicee application.

Afterwards, ESA also decided to participate acting as an Application Provider with the Mosaicing application.

7. Earth Observation Exploitation Platform

OGC EO Apps Pilot project invited Earth Observation platform operators to implement the OGC Earth Observation Applications Pilot architecture as it has been defined in previous IP initiatives.

Spacebel acts in this pilot as a platform provider with the EORegions! platform.

7.1. Architecture Technical Aspects

OGC Testbed activities in Testbed-13, Testbed-14, and Testbed-15 have contributed to the definition of an EO platform architecture that allows the deployment and execution of applications close to the physical location of the source data. The goal is to minimize data transfer between data repositories and application processes.

The main Engineering Reports describing the architecture are listed below:

  • OGC 18-049r1 - Testbed-14: Application Package Engineering Report [2]

  • OGC 18-050r1 - Testbed-14: ADES & EMS Results and Best Practices Engineering Report [3]

The figure below shows an overview of the Testbed-14 architecture showing the main components (EMS and ADES).

tb architecture
Figure 1. Testbed 14 Architecture Overview

7.1.1. Application Deployment and Execution Service

Applications developed by application providers are deployed using the Application Deployment and Execution Service (ADES). OGC Web Processing Service 2.0 is the standard interface used to expose all ADES operations (application discovery and execution management). Some operations rely on an OGC WPS extension:

  • Transactional extension (register and unregister processes);

  • Quoting extension (quote an execution before submission);

  • Billing extension (provide payment information);

  • Visibility extension (authorization mechanism superseded by policy enforcement point architecture).

Web Processing Service (WPS) Specification

OGC WPS (version 2.0) provides a standard remote interface to encapsulate any processing application. WPS is a front-end interface and the implementation typically delegates the execution of application to a back-end server.

Concretely, a WPS server might support many architectures including the following examples:

  • Java program executed by the backend Java Virtual Machine (JVM);

  • Python library submitted to the configured interpreter;

  • Docker container distributed to a Kubernetes cluster node;

  • Business Process Execution Language (BPEL) workflow managed by a remote workflow engine;

Figure 2. WPS Possible Backends
Discovery and Application Descriptor

The WPS defines a well-known stable interface contract (i.e. an API) but intends to support a wide range of geospatial applications offering different functions and various inputs & outputs.

In order to discover the registered applications, the WPS provides the operations GetCapabilities and DescribeProcess. Processes are modeled in a Process Description, i.e. an application descriptor capturing all details needed by the client to interact and execute the application.

Concretely, the Process Description helps to generate a customized user interface for composing the execution request, integrate adapted tools (EO data browser), enforce constraints and to potentially automate the executions. The descriptor provides various relevant information such as the following properties:

  • Name, description, cardinality of inputs;

  • Input types (literal, file, EO product);

  • Input constraints (temporal, spatial, etc.) and relations (e.g. same relative orbit);

  • Product catalogue or database endpoints;

  • Billing related information;

  • Output data and metadata types;

  • Etc.

Figure 3 illustrates an example of the view generated from an application descriptor in a WPS user interface. The integration of a tailored selection of products is shown for the first input field.

wps discovery
Figure 3. WPS User Interface
Deployment and Application Package

Since the early days of WPS 1.0, the need for an operation for registering new application emerged. In 2008, the first draft of a transactional extension was submitted for harmonizing the deployment of processes. The operations deploy and undeploy have been then updated and improved continuously as illustrated on figure below.

Figure 4. WPS Transactional Specification History

As described in the OGC 18-036r1 WPS-T Engineering Report, the deploy request essentially consists of the list of execution units (files, libraries) composing the application [4]. For each kind of supported application (BPEL workflow, Java library, CWL package), the exact format of items should be defined and is identified using the Application Package reference.

From the provided bundle of files composing the application, the WPS needs to generate the corresponding application descriptor described above, excepted if the document is provided.

As illustrated in Figure 5 below, WPS might expose inputs diverging from the backend process. For example, the WPS could expose some Catalogue search parameters, while the search results (EO data) are actually consumed by the process.

Figure 5. WPS Discrepancy between Internal and External Interfaces

7.1.2. Execution Management Service

An EMS is defined as the Thematic Exploitation Platform (TEP) processing frontend ahead of multiple platform (Mission Exploitation Platforms).

The end-users of the TEP register their applications through the EMS, and the target platform used to execute a process is selected at runtime based on the location of the data to be processed. The EMS also provides support for processing chains: workflows can be deployed as a CWL (Common Workflow Language) document. The Business Process Model and Notation (BPMN) alternative has also been explored in Testbed-14 [5].

The current EMS interface also implements the WPS standard, and thus the component acts like a proxy aggregator.

7.1.3. Supported Application Packages

The application is registered in a package providing the execution unit and optionally the application descriptor. Testbed architecture currently defines the following Application Package profiles:

  • Docker container

  • CWL Workflow

Examples are provided in OGC Testbed-14: Application Package Engineering Report [2].

Docker Container Application Package

The Docker Application Package is defined as follows:

  • Process Description (with the reference of the CWL);

  • CWL Descriptor based on the CommandLineTool class and properties itemSeparator, position, prefix, separate and glob;

  • Execution Unit: Docker Image reference.

CWL Workflow Application Package

The CWL Workflow Application Package is defined as follows:

  • Process Description;

  • Execution Unit: CWL workflow describing the ADES calls using the application identifier.

7.2. Spacebel Platform Overview

As a "Platform Provider" (WP2), Spacebel is offering the "Geospatial Exploitation Platform" (GEP) as a facility to be used for the Pilot, and its main instantiation called “EO Regions!”.

7.2.1. EO Regions! Context

The EO Regions! Platform was originally funded by the Walloon region and is being operated by Spacebel. EO Regions! is the Walloon hub of the European EUGENIUS network and is recognised as Copernicus Relay for Wallonia by the European Commission for the promotion and support to users of Copernicus data. The main user facing Web Portal is available at and provides more background information about the platform and its objectives.

The Walloon regional government has recently confirmed its industrial policy and related strategy for exploiting EO data in its Déclaration de Politique Régionale (DPR) 2019-2024 , and the intention to implement the Walloon part of the Luxembourg/Walloon Collaborative Ground Segment (CGS). EO Regions! and its applications (currently hosted on Google Cloud Platform) will be hosted on the CGS (expected to be located in Transinne-Belgium) when it becomes available.

The EO Regions! platform mostly relies on a set of components known internal as "Geospatial Exploitation Platform" and described in section below.

7.2.2. GEP components

The main components of the EO Regions! Platform are described below:

  • The ADES Processing Service handles all aspects pertaining to the deployment and execution of EO applications on a cloud environment as designed by OGC. The component implements the Web Processing Service Transactional specification (WPS-T) with REST/XML bindings and relies on a Kubernetes cluster for the execution of Docker containers.

  • The ADES Scheduler component automates repeated executions triggered by a set of events. For example, any processes can be executed with a set of defined parameters (e.g. a given area of interest) either on regular basis (daily, weekly, etc.), or based on the availability of new source tile.

  • The Ingestion Data Manager essentially registers and manages the geospatial data generated by the processes by copying (if needed) the new data resources into the File Store and registering metadata into the OpenSearch Catalogue. The Data Manager is able to discover the Landsat, Sentinel-2 and NEXRAD data available on the Google Cloud Platform and can also fetch remote data shared using a set of protocols (HTTP, FTP, etc.).

  • The Catalogue registers metadata of all products available locally and makes them discoverable using an OpenSearch interface (OGC 13-026r8). The resources may be input data (e.g. from a satellite sensor) or data generated by the applications. The metadata includes the references with the protocols available for the given resources.

  • The Data Resources File Store holds the EO resources and generated products in a hybrid set of logical storages. The supported storage systems are NFS, Object Storage (virtually abstracted as a file storage using FUSE) and HDFS.

  • The Data Access Service provides standardized access Data Resources File Storage (additionally to the POSIX File System protocol). The Data Access Service currently only supports HTTP.

7.3. Application Developer Perspective

The following sections cover the development and execution aspects from a user point of view.

7.3.1. Application Development

The nominal scenario for developing an application that can be registered and executed on an EO platform consists of the following steps:

  1. Implement a compatible application

  2. Build a Docker Image

  3. Register the Docker Image

  4. Describe the Application

Write the application

The assumptions below must be satisfied for ensuring compatibility with the EO platform:

  • Executable can be started from a UNIX command (e.g. a bash script).

  • Inputs (i.e. parameters or EO products locations) can be provided to the app either using environment variable or through UNIX command arguments.

  • Output files should must be written either in the working directory when starting the application, or in the location indicated in a configurable environment variable.

The example below is a very simple bash script which receives as input a file (indicated in the $WPS_INPUT_INPUT1 environment variable) and writes a compressed archive of the input in the specified output location (received in $WPS_OUTPUT_OUTPUT1).

set -x # display environment variables (for information only)
echo "Starting this dummy processing chain - Hello World !"
echo "Display input directory location"
echo "As a simple demo, let's archive the input directory into a zip file"
tar -czvf "$WPS_OUTPUT_OUTPUT1" ${WPS_INPUT_INPUT1//,/ }
echo "Chain is finished, thank you"
Build the Docker Image

A major benefit of Docker containers is that the entire environment is embedded in the application package. Indeed, the environment is described in a formal description file named a Dockerfile. The procedure for building containers using the very popular Docker technology is explained at

In the example below, the container is based on a java:8-jre image, to which the bash script shown above is added, and finally the script is declared as the container entrypoint (using CMD command).

FROM java:8-jre
USER root
# Depending on the docker host umask , the following line is optional so please keep it
RUN  chmod a+x /

In order to build a docker image locally (which requires a Docker engine installed), the following command is used:

docker build -t my-container-name
Register the Docker Image

A free and easy way to share the Docker image is to use the Docker Hub, i.e. the official Docker repository. In order to register the Docker container, a Docker Hub account is needed. The following command authenticates the user:

docker login --username=yourhubusername

Docker images are then pushed using the following command (e.g. we called our container “copy-processing-chain”):

docker tag copy-processing-chain yourhubusername/copy-processing-chain:latest
docker push yourhubusername/copy-processing-chain:latest

Note that the Spacebel GEP platform provides a private registry for hosting the developed applications.

Application Package and Application Descriptor

The Application Package provides the execution units composing the application and all information needed by the platform to interact with the application. In the scope of a containerized application, this includes:

  • The Docker Image Reference

  • The application command-line interface (how inputs can be submitted, and where outputs can be collected)

  • An optional Application Descriptor for advanced users (further described in the report).

Note that the Spacebel platform provides some UI tools that help the developer generate the Application Package items.

7.3.2. Deployment and Execution


When accessing the Application Provider console of the Spacebel EO Regions! platform, the sign in page is displayed. The user needs to authenticate through the OpenAM Single Sign On facility.

Figure 6. OpenAM Sign In

The initial displayed view is the Applications tab.


The Applications tab displays the list of available processes.

Figure 7. Spacebel Platform - Applications List

Clicking on a specific process displays an execution form for submitting an execution.


The deployment tab allows the user to register a new application. From the tab Upload Package, the user can provide the CWL or the Process Description (both supported) of the application.

The Wizard tab helps the user to build automatically the Application Package if the Application Descriptor is not available.

Figure 8. Spacebel Platform - Deployment Wizard

The Execution manager displays a generated form including the fields declared in the Application Package. The execution form ensures all constraints and integrates all the relevant tools depending on the discovered content of the Process Description.

Figure 9. Spacebel Platform - Execution Manager

The fields that are declared as File or Directory (Complex Data) can be either:

  • Local computer resources: uploaded from the upload button;

  • Remote resources: provided by an HTTP reference (which will be staged in by the platform);

  • Platform EO resources: provided by an NFS reference found by browsing the catalogue.

Once the input values have been set, the user needs to click on the Execute button. The Execution Results view can be selected to monitor the running execution.

From the Execution Management view, the "magnifier" button opens a dedicated browser tab that allows to search for input EO products.

The first step allows the user to select the collection of interest: either native Copernicus products, or products generated through the execution of other processes.

The second step allows to search for products by providing the required filters (orbit number, time period, tile identifier, etc.) or drawing an area of interest.