5.1.  Identifiers

The normative provisions in this document are denoted by the URI

http://www.opengis.net/spec/eoap-bp/1.0

All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.

5.2.  Abbreviated terms

• API Application Programming Interface

• COG Cloud Optimized GeoTIFF

• CWL Common Workflow Language

• EO Earth Observation

• EP Exploitation Platform

• GDAL Geospatial Data Abstraction Library

• IW Interferometric Wide

• JSON JavaScript Object Notation

• OGC Open Geospatial Consortium

• OS Operating System

• OWS OGC Web Services

• REST Representational State Transfer

• SAFE Standard Archive Format for Europe

• SLC Single Look Complex

• SNAP Sentinel Application Platform toolbox

• STAC SpatioTemporal Asset Catalog

• TBD To Be Determined

• TIFF Tagged Image File Format

• URL Uniform Resource Locator

• YAML YAML Ain’t Markup Language

6.1.  Introduction

In recent years, Platforms for the Exploitation of Earth Observation (EO) data have been developed by public and private companies in order to foster the usage of EO data and expand the market of Earth Observation-derived information. The domain is composed of platform providers, service providers who use the platform to deliver a service to their users, and data providers. The availability of free and open data (e.g. Copernicus Sentinel), together with the availability of affordable computing resources, creates an opportunity for the wide adoption and use of EO data in a growing number of fields in our society.

An EO exploitation platform is a collaborative virtual work environment providing the mechanisms to deliver EO data and the tools, processors, and ICT resources required to work with them, through one coherent set of interfaces. A fundamental principle of the platform operations concept is to move the EO data processing service’s user to the data and tools, as opposed to downloading, replicating, and exploiting data ‘at home’. Furthermore, platforms offer an environment which takes care of all data processing orchestration tasks and the availability of scalable computational resources offered by the Cloud shortens the time to market of applications.

In this scope, OGC Testbeds 13-16 initiated the drafting of an architecture to allow the deployment and execution of externally developed applications on Earth Observation (EO) data and processing platforms. During the OGC Innovation Program initiative OGC Earth Observation Applications Pilot, conducted between December 2019 and July 2020 (OGC 20-042, OGC 20-073), the participants explored and assessed the existent draft specifications, which addressed both application description and discovery, APIs for deployment, execution, and result access, as well as specifications for service chaining and workflow building. This document summarizes their findings and defines a Best Practice to package and deploy Earth Observation Applications in an Exploitation Platform.

In summary, the architecture as defined by the OGC Earth Observation Applications Pilot targets the following requirements:

• Decouple application developers from exploitation platform operators and from application consumers

• Allow application developers to make their applications available on any number of platforms with minimal modifications

• Allow application developers to focus on application development by minimizing platform specific particularities

• Enable exploitation platforms to virtually support any type of packaged EO application

The figure below provides a high-level view of the interactions between the actors and the architecture.

Figure 1 — Application developers and application consumers interacting with the cloud platform

Application developers on the left side develop their application and make it available in the form of an Application Package referencing one or more container images. The application developers register their application in a platform making it available to users.

Application consumers can discover applications available in the platform, request their execution with a given set of input products and parameters and obtain the resulting products in their platform user space.

6.2.  Earth Observation Applications

Earth Observation Applications typically offer functions that perform data operations like processing / reprocessing, projection, visualization or analysis. The applications can be written in a variety of coding languages (e.g. Python, R, Java, C++, C#, shell scripts) and make use of specific software libraries (e.g. SNAP, GDAL, Orfeo Toolbox).

In the context of this Best Practice, the application is treated as a black-box that according to its application design pattern must comply with data stage-in and data stage-out mechanisms defined. Two main design patterns are identified: fan-in and fan-out. An application can combine these patterns in the nodes of a Directed Acyclic Graph (DAG).

6.2.1.  Data-driven application with a fan-in application pattern

The data driven application fan-in pattern refers to the execution of a data processing function that aggregates several input products.

The platform application accesses a list of input products, stages the input products making them available to the application execution block.

Figure 2 — Data-driven application with fan-in input references where an application processes the aggregates of n-input EO products

An application following this pattern must take in consideration that it will be invoked once with all the input products and is expected to create one output (but not necessarily a single file).

6.2.2.  Data-driven application with a fan-out application pattern

The data driven application fan-out pattern refers to the execution of a data processing function that processes concurrently several products generating independent output for each input.

The platform application loops from a list of input products, stages each of the individual products making it available to the application execution block. The platform can apply different strategies to parallelize the execution of each individual product.

Figure 3 — Data-driven application with fan-out input references where an application processes several input EO products independently.

6.2.3.  Staging Input and Output EO Products

EO product files come in different formats (e.g. GeoTIFF, HDF5, SAFE) and might have sub-items (e.g. metadata, bands, masks) that can be encoded in the same file or follow a given folder structure.

For example, SENTINEL-2 products are made available to users in the SENTINEL-SAFE format, including image data in JPEG2000 format, quality indicators (e.g. defective pixels mask), auxiliary data and metadata. The SAFE format wraps a folder containing image data in a binary data format and product metadata in XML. A SENTINEL-2 product refers to a directory folder that contains a collection of information that can include several files like seen in the next figure.

Figure 4 — SENTINEL-2 product physical format

A main concern application developers face is the different approaches through which the products are made available (i.e. stage-in) to the applications. For example, applications might find the same exact folder structure and return the folder root or the main XML manifest file or have the folder structure compressed in a single archive file.

In general, the onus of navigating the input folder directory and programmatically reacting to how the file was staged-in by the platform is on application and the application developer needs to consider all possible cases when developing their read routines.

Conversely, the outputs of the application are fully managed by the developer that places the resulting files in an output directory. The only information the platform might receive about the output files is the file media type (formerly known as “MIME-type”) and is often missing critical information like spatial footprint, sub-items (e.g. masks, bands) and additional metadata (e.g. ground sample distance, orbit direction).

A good solution to represent the data manifest for input and output products is brought by the SpatioTemporal Asset Catalog (STAC).

The STAC specification standardizes the way geospatial assets are exposed online and queried. A ‘spatiotemporal asset’ is any file that represents information about the earth captured in a certain space and time (e.g. satellites, planes, drones, balloons).

The STAC specification defines several objects:

• STAC Catalog: STAC Catalog is a collection of STAC Items or other STAC Catalogs (sub-catalogs). The division of sub-catalogs is transparently managed by links to ease online browsing.

• STAC Collection: extends the STAC Catalog with additional fields to describe a whole set of STAC Items that share properties and metadata. STAC Collections are meant to be compatible with OGC API — Features Collections (OGC 17-069r3).

• STAC Item: a GeoJSON Feature with additional fields (e.g. time, geo), links to related entities and STAC Assets.

• STAC Asset: is an object that contains a link to data associated with the STAC Item that can be downloaded or streamed (e.g. data, metadata, thumbnails) and can contain additional metadata. Similar to atom:link it has properties like href, title, description, type and roles; but, most significantly, it allows relative paths.

Most importantly the STAC specification can be implemented in a completely ‘static’ manner as flat local files located near the data enabling the application to access products assets (e.g. JPEG 2000 band file, auxiliary data, browse) with a relative path (something that was not possible using OpenSearch as defined by OGC 13-026r8, OGC 13-032r8).

This Best Practice selected a STAC Catalog with STAC Item files as the data manifests format, for application that require staging input data and/or output results.

6.3.  Application Package

The Application Package is a document that describes the data processing application by providing information about the parameters, software item, executable, dependencies and metadata. This file document ensures that the application is fully portable among all supporting processing scenarios and supports automatic deployment in a Machine — To — Machine (M2M) scenario. Most importantly, the Application Package information model allows the deployment of the application as an OGC API — Processes (OGC 18-062) compliant web service.

The Application Package includes the following information:

• Reference to the executable block that implements the Application functionality

• Description of its input/output interface

The Application Package uses the Common Workflow Language (CWL) Workflow Description specification as an encoding to describe the Application, its parameters, command-line tools, their runtime environments, their arguments and their invocation within containers.

6.4.  Platform

An Earth Observation Exploitation Platform provides interfaces, processing functions, tools and processing services invoked individually or utilized in workflows. Developers are able to test and execute their own applications, register them into the platform and make them available for exploitation by other users also individually or in their own workflows.

Connected from Client Portals, users of Earth Observation applications are able to find and exploit on the Platform the services matching their needs, out of a large offer gathered from multiple contributors.

Interfaced with Cloud Computing providers, Earth Observation Exploitation Platform executes the data processing tasks requested by users and retrieves the information produced for delivery back to the processing requester.

To support the application integration activities, a platform might also provide developers with an environment where they can integrate, build, test & debug their applications as part of their business use case. The ultimate goal of such an environment is to produce an Application Package. However, these steps are outside the scope of this Best Practice that focuses on the platform as the environment where the data processing application is registered and executed.

6.4.1.  Platform Architecture

This best practice focuses on the scenario where an application is directly packaged as an Application Package, registered in a Platform and made available as an implementation of OGC API — Processes. The Web Service allows end-user portals and B2B client applications to pass processing parameters, trigger on-demand or systematic data processing requests and establish the data pipeline to retrieve the information produced.

Figure 5 — Architecture overview of the Application execution in a Platform

In the context of this Best Practice, the main responsibilities of the Platform are to:

• Validate and accept an application deployment request (OGC API — Processes Transaction Extension)

• Validate and accept an execution request (OGC API — Processes)

• Submit the process execution to the processing cluster (CWL)

• Monitor the process execution (OGC API — Processes)

• Retrieve the processing results (OGC API — Processes)

For applications that require staged EO products and/or generate products that need to be staged-out the platform is responsible for the EO Product data flow management operations for:

• Data stage-in of the process input EO Product.

• Data stage-out of the process outputs.

7.1.  Overview

An Application that complies with the Best Practice for Earth Observation Application Package needs to be:

• Executable as a command-line tool.

• Delivered in a container image with all the necessary software, libraries and configuration files.

This section described the best practice for the command line tools, how to consider input data, output data and how to create a container.

7.2.  Command Line

The Application is executed as a command-line interface (CLI) tool that runs as a non-interactive executable program: it receives input arguments, performs a computation, and terminates after producing some output.

The Application can have any number of command-line arguments.

When executed, the Application working directory is also the Application output directory. Any file created by the Application should be added under that directory.

#!/bin/bash

if [ "$#" -lt 3 ]
then
  echo "Usage: provide file, bbox and proj"
  exit 1
fi

# file to process
file=$1
# bbox processing argument
bbox=$2
# EPSG code used to express bbox coordinates
proj=$3

gdal_translate \
      -projwin \
      "$( echo $bbox | cut -d ',' -f 1)" \
      "$( echo $bbox | cut -d ',' -f 4)" \
      "$( echo $bbox | cut -d ',' -f 3)" \
      "$( echo $bbox | cut -d ',' -f 2)" \
      -projwin_srs \
      ${proj} \
      ${file} \
      cropped.tif

If the EO Products are already staged on the local computer this command-line could be executed with the parameters.

computer:~ user$ crop "S2B_53HPA_20210723_0_L2A/B02.tif" "136.522,-36.062,137.027,-35.693" "EPSG:4326"

7.3.  Container

The environment, libraries, binaries and configuration files necessary to execute the command-line tools need to be bundled in a container image.

The example below shows how Docker, one of the available container engine solutions to deliver software in containers, defines all the necessary commands to assemble an image.

FROM osgeo/gdal

RUN apt update && \
    apt-get install -y jq

ADD functions.sh /functions.sh

ADD crop /usr/bin/crop

RUN chmod +x /usr/bin/crop

Build the docker image with:

docker build -t crop_docker:0.1 .

Test the CLI with:

docker run --rm -it crop_docker:0.1 crop

#!/bin/bash

if [ "$#" -lt 3 ]
then
  echo "Usage: provide file, bbox and proj"
  exit 1
fi

## generic STAC functions
source /functions.sh

## processing arguments
in_dir=$1 # folder where the EO product is staged-in
bbox=$2 # bbox processing argument
proj=$3 # EPSG code used to express bbox coordinates

## Read the input STAC Catalog
catalog="${in_dir}/catalog.json"

# get the item path
item=$( get_items ${catalog} )

# get the B02 asset href (local path)
asset_href=$( get_asset ${item} B02 )

gdal_translate \
      -projwin \
      "$( echo $bbox | cut -d ',' -f 1)" \
      "$( echo $bbox | cut -d ',' -f 4)" \
      "$( echo $bbox | cut -d ',' -f 3)" \
      "$( echo $bbox | cut -d ',' -f 2)" \
      -projwin_srs \
      ${proj} \
      ${asset_href} \
      cropped.tif

{
  "id": "catalog",
  "stac_version": "1.0.0",
  "links": [
    {
      "type": "application/geo+json",
      "rel": "item",
      "href": "S2B_53HPA_20210723_0_L2A/S2B_53HPA_20210723_0_L2A.json"
    }
  ],
  "type": "Catalog",
  "description": "Root catalog"
}

{
  "stac_version": "1.0.0",
  "stac_extensions": [ "eo", "proj", "view"],
  "type": "Feature",
  "id": "S2B_53HPA_20210723_0_L2A",
  "geometry": {
    "type": "Polygon",
    "coordinates": [ [
        [ 136.11273785955868, -36.22788818051635],[ 136.09905192261127, -35.238096451039816],[ 137.30513468251897, -35.22113204961173],[ 137.33381497932513, -36.21029815477051], [ 136.11273785955868, -36.22788818051635]
      ] ]
  },
  "properties": {
    "datetime": "2021-07-23T00:57:07Z",
    "platform": "sentinel-2b",
    "constellation": "sentinel-2",
    ...
  },
  "bbox": [ 136.09905192261127, -36.22788818051635, 137.33381497932513, -35.22113204961173],
  "assets": {
    ...
    "B02": {
      "type": "image/tiff; profile=cloud-optimized; application=geotiff",
      "roles": [ "data" ],
      "title": "Band 2 (blue)",
      "href": "B02.tif",
      "gsd": 10,
      "eo:bands": [
        {
          "name": "B02",
          "common_name": "blue",
          "center_wavelength": 0.4966,
          "full_width_half_max": 0.098
          }
      ],
      "proj:shape": [ 10980, 10980 ],
      "proj:transform": [ 10,  0, 600000, 0, -10, 6100000, 0, 0, 1 ],
      "file:size": 206117177
    },
    ...
  },
  "links": [
    {
      "type": "application/json",
      "rel": "canonical",
      "href": "https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/53/H/PA/2021/7/S2B_53HPA_20210723_0_L2A/S2B_53HPA_20210723_0_L2A.json"
    },
    {
      "rel": "parent",
      "href": "../catalog.json"
    }
  ]
}

# parse the catalog.json file and get the first STAC item
item="$( jq -r '.links | select(.. | .rel? == \"item\")[0].href' catalog.json)"

# get asset B02 href
asset_href=$( dirname ${item} )/$( cat ${item} | jq -r ".assets.B02.href" )

gdal_translate ${asset_href} output.png

7.4.  EO Products as Output Data

An Application that creates EO product files that need to be staged-out must also create in the output directory a STAC Catalog that enumerates and documents the produced files.

Below is a catalog.json produced by the Application with the one result referenced as a STAC Item.

{
  "id": "catalog",
  "stac_version": "1.0.0",
  "type": "catalog",
  "description": "Result catalog",
  "links": [
    {
      "type": "application/geo+json",
      "rel": "item",
      "href": "result-item.json"
    }
  ]
}

The STAC Item files contain the corresponding STAC Assets with the results of the processing. Each STAC Asset contains a reference to the associated data (e.g. data, metadata, thumbnails).

{
  "id": "item_id",
  "stac_version": "1.0.0",
  "type": "Feature",
  "bbox": [ 136.522, -36.062, 137.027, -35.693
  ],
  "geometry": {
    "type": "Polygon",
    "coordinates": [
      [
        [ 136.522, -36.062],
        [ 137.027, -36.062],
        [ 137.027, 137.027],
        [ 136.522, 137.027],
        [ 136.522, -36.062]
      ]
    ]
  },
  "properties": {
    "datetime": "2021-07-23T00:57:07Z",
    "gsd": 10
  },
  "assets": {
    "B02": {
      "role": [ "data"],
      "href": "./cropped.tif",
      "type": "image/tiff",
      "title": "Cropped B02 band"
    }
  }
}

The STAC Catalog created by the Application must include metadata elements defining properties as the start time & end time or the geographical footprint. These two elements are the minimum set of metadata elements to enable their discovery but depending on the context, the application can convey further elements.

It is assumed that all output files not referenced in the STAC Catalog local file are not relevant to the process and can be discarded by any subsequent action and thus not staged out.

Please note that, as with the input data, nothing hampers the Application to be a wrapper command-line tool that, after executing a command-line, creates the STAC Catalog referencing the output files.

The example below complements the previous examples with the creation of the STAC Catalog for the data output manifest.

#!/bin/bash

if [ "$#" -lt 3 ]
then
  echo "Usage: provide file, bbox and proj"
  exit 1
fi

## generic STAC functions
source /functions.sh

## processing arguments
in_dir=$1 # folder where the EO product is staged-in
bbox=$2 # bbox processing argument
proj=$3 # EPSG code used to express bbox coordinates

## Read the input STAC Catalog
catalog="${in_dir}/catalog.json"

# get the item path
item=$( get_items ${catalog} )

# get the B02 asset href (local path)
asset_href=$( get_asset ${item} B02 )

gdal_translate \
      -projwin \
      "$( echo $bbox | cut -d ',' -f 1)" \
      "$( echo $bbox | cut -d ',' -f 4)" \
      "$( echo $bbox | cut -d ',' -f 3)" \
      "$( echo $bbox | cut -d ',' -f 2)" \
      -projwin_srs \
      ${proj} \
      ${asset_href} \
      cropped.tif

## result as STAC
# get the properties from the input STAC item
# as these are the same for the output STAC item
datetime=$( get_item_property ${item} "datetime" )
gsd=$( get_item_property ${item} "gsd" )

# initialise a STAC item
init_item ${datetime} "${bbox}" "${gsd}" > result-item.json

# add an asset
add_asset result-item.json "B02" "./cropped.tif" "image/tiff" "Cropped B02 band"

# initialise the output catalog
init_catalog > catalog.json

# add the item to the catalog
add_item catalog.json result-item.json

7.5.  Requirement Classes

8.1.  Overview

A Package that complies with the Best Practice for Earth Observation Application Package needs to:

• Be a valid CWL document with a single Workflow Class and at least one CommandLineTool Class

• Define the command-line and respective arguments and container for each CommandLineTool

• Define the Application parameters

• Define the Application Design Pattern

• Define the requirements for runtime environment

The Workflow class steps field orchestrates the execution of the application command line and retrieves all the outputs of the processing steps.

8.2.  CWL Document

The CWL Document references the Application parameters with the class Workflow and the command lines tools and arguments with the CommandLineTool classes.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...

- class: CommandLineTool
  id: crop-cl
  ...

8.3.  Command-Line Tool

As stated previously, the command-line tool is a non-interactive executable program that reads some input, performs a computation, and terminates after producing some output.

The CommandLineTool class defines the actual interface of the command-line tool and its arguments according to the CWL CommandLineTool standard.

The CWL explicitly supports the use of software container technologies, such as Docker or Singularity, to enable portability of the underlying analysis tools. The Application Package needs to explicitly provide for each command-line tool the container requirements defining the container image needed.

The field DockerRequirement indicates that the component should be run in a container, and specifies how to fetch or build the image.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...

- class: CommandLineTool
  id: crop-cl
  requirements:
    DockerRequirement:
      dockerPull: docker.io/terradue/crop_container
  baseCommand: crop
  arguments: []
  inputs:
  ...
  outputs:
  ...

The field inputs defines the list of input parameters of the command-line that control how to run the tool. Each parameter has an id for the name of parameter, and a type field describing what types of values are valid for that parameter (e.g. string, int, double, null, File, Directory, Any). Additionally, if there are command-line bindings not directly associated with input parameters (e.g. fixed values or environment run-time values), the field arguments can also be used.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...

- class: CommandLineTool
  id: crop-cl
  ...
  baseCommand: crop
  arguments: []
  inputs:
    ...
    bbox:
      type: string
      inputBinding:
        position: 2
    epsg:
      type: string
      inputBinding:
        position: 3
  outputs:
  ...

When the command-line is executed under CWL, the starting working directory is the designated output directory. The underlying tool or script records its results in the form of files created in the output directory.

All the outputs of the command line tool are retrieved at this level.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...

- class: CommandLineTool
  id: crop-cl
  ...
  outputs:
    cropped_tif:
      outputBinding:
        glob: .
      type: Directory
...

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...

- class: CommandLineTool
  id: crop-cl
  requirements:
    DockerRequirement:
      dockerPull: docker.io/terradue/crop_container
  baseCommand: crop
  arguments: []
  inputs:
    product:
      type: Directory
      inputBinding:
        position: 1
    band:
      type: string
      inputBinding:
        position: 2
    bbox:
      type: string
      inputBinding:
        position: 3
    epsg:
      type: string
      inputBinding:
        position: 4
  outputs:
    cropped_tif:
      outputBinding:
        glob: .
      type: Directory
...

8.4.  Application

The CWL Workflow class defines the Application as an analysis task represented by a directed graph describing a sequence of operations that transform an input data set to output.

The Workflow class includes four basic blocks: identification, inputs, steps and outputs.

For the identification block, the Workflow class supports the definition of a unique identifier (id), a short human-readable title (label) and a long human-readable description (doc) of the Application.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  label: Sentinel-2 band crop
  doc: This application crops a band from a Copernicus Sentinel-2 product using GDAL
  ...

- class: CommandLineTool
  id: crop-cl
  ...

For the inputs, the Workflow class supports the definition of the input parameters of the process. Each input parameter has a corresponding identifier (the field’s name), title (label), abstract (doc) and a type (type) that is mandatory according to the CWL Workflow specification.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...
  inputs:
    ...
    bbox:
      type: string
      label: bounding box
      doc: Area of interest expressed as a bounding box
    proj:
      type: string
      label: EPSG code
      doc: Projection EPSG code for the bounding box
      default: "EPSG:4326"
  steps:
  ...
  outputs:
  ...
  - class: CommandLineTool
  id: crop-cl
  ...

The workflow is managed by the steps field of the Workflow class that links the corresponding parameters with arguments of the command-line class defined in the previous section.

cwlVersion: v1.0
$graph:
- class: Workflow
  label: Sentinel-2 band crop
  doc: This application crops a Sentinel-2 band
  id: s2-cropper
  inputs:
    product:
    ...
    band:
    ...
    bbox:
    ...
    proj:
    ...
  steps:
    node_crop:
      run: "#crop-cl"
      in:
        product: product
        band: band
        bbox: bbox
        epsg: proj
      out:
        - cropped_tif
  outputs:
  ...

- class: CommandLineTool
  id: crop-cl
  ...
  inputs:
    product:
       ...
    band:
       ...
    bbox:
       ...
    epsg:
       ...
  outputs:
  ...

The previous workflow can be visualized as shown in the next figure.

Figure 6 — Workflow diagram for the a simple Application with four inputs parameters and one execution block

For the outputs, the Workflow class includes the outputs section. This is a list of output fields where each field consists of an identifier and a data type.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...
  inputs:
  ...
  steps:
  ...
  outputs:
    results:
      outputSource:
      - node_crop/cropped_tif
      type: Directory

  - class: CommandLineTool
  id: crop-cl
  ...

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  label: Sentinel-2 band crop
  doc: This application crops a Sentinel-2 band
  inputs:
    product:
      type: Directory
      label: Sentinel-2 inputs
      doc: Sentinel-2 Level-1C or Level-2A input reference
   band:
      type: string
      label: Sentinel-2 band
      doc: Sentinel-2 band to crop (e.g. B02)
    bbox:
      ...
    proj:
      ...
  steps:
    node_crop:
      run: "#crop-cl"
      in:
        product: product
        band: band
        bbox: bbox
        epsg: proj
      out:
        - cropped_tif
  outputs:
    results:
      outputSource:
      - node_crop/cropped_tif
      type: Directory

- class: CommandLineTool
  id: crop-cl
  ...

{
  "id": "catalog",
  "stac_version": "1.0.0",
  "type": "catalog",
  "description": "Result catalog",
  "links": [
    {
      "type": "application/geo+json",
      "rel": "item",
      "href": "result-item.json"
    }
  ]
}

{
  "id": "item_id",
  "stac_version": "1.0.0",
  "type": "Feature",
  "bbox": [ 136.522, -36.062, 137.027, -35.693 ],
  "geometry": {
    "type": "Polygon",
    "coordinates": [
      [ [ 136.522, -36.062 ],
        [ 137.027, -36.062 ],
        [ 137.027, -35.693 ],
        [ 136.522, -35.693 ],
        [ 136.522, -36.062 ] ]
    ]
  },
  "properties": {
    "datetime": "2021-07-23T00:57:07Z",
    "gsd": 10
  },
  "assets": {
    "B02": {
      "role": [ "data" ],
      "href": "./cropped.tif",
      "type": "image/tiff",
      "title": "Cropped B02 band"
    }
  }
}

8.5.  Application Pattern

The fan-in application design pattern, shown in the previous examples, is the simplest case where all the EO inputs are used for the single processing. This pattern is the default behavior of a CWL workflow execution step.

For the fan-out application design pattern to be expressed in CWL, it is necessary to use the ScatterFeatureRequirement requirement. This requirement tells the CWL runner that the application will run multiple times over a list of inputs. The workflow then takes the input(s) as an array and will run the specified step(s) on each element of the array as if it were a single input.

For the fan-out design pattern the Workflow class needs to change the type of input to an array (i.e. a string with square brackets string[]), and add a new requirement with the ScatterFeatureRequirement field to change the specific step as shown below where the input parameter band was changed to bands for the fan-out behavior.

cwlVersion: v1.0
$graph:
- class: Workflow
  label: Sentinel-2 band crop
  doc: This application crops a Sentinel-2 band
  id: s2-cropper

  requirements:
  - class: ScatterFeatureRequirement

  inputs:
    ...
    bands:
      type: string[]
      label: Sentinel-2 bands
      doc: Sentinel-2 list of bands to crop
    ...
  outputs:
    results:
      outputSource:
      - node_crop/cropped_tif
      type: Directory[]

  steps:
    node_crop:
      run: "#crop-cl"
      in:
        product: product
        band: bands
        bbox: bbox
        epsg: proj
      out:
        - cropped_tif
      scatter: band
      scatterMethod: dotproduct

- class: CommandLineTool
  id: crop-cl
  ...
  inputs:
    ...
    band:
      type: string
      inputBinding:
        position: 2
    ...
  outputs:
    ...
...

In the definition above the fan-out pattern is applied to the bands workflow field is mapped to a band parameter at step level.

The field scatter is used to define which step input parameter is scattered in the workflow step requirements and the fan-out method is defined with the scatterMethod field. Its value is one of: dotproduct, nested_crossproduct, or flat_crossproduct:

• dotproduct specifies that each of the input arrays are aligned and one element taken from each array to construct each job. It is an error if all input arrays are not the same length.

• nested_crossproduct specifies the Cartesian product of the inputs, producing a job for every combination of the scattered inputs. The output must be nested arrays for each level of scattering, in the order that the input arrays are listed in the scatter field.

• flat_crossproduct specifies the Cartesian product of the inputs, producing a job for every combination of the scattered inputs. The output arrays must be flattened to a single level, but otherwise listed in the order that the input arrays are listed in the scatter field.

The full Application Package file is included in Annex D.

8.6.  Extended Workflows

The Workflow class can orchestrate one or more command-line applications in a directed acyclic graph describing a sequence of operations that transform input data sets to output.

The example below shows a new tool, called composite that creates a RGB image from three inputs for the red, green and blue channels.

cwlVersion: v1.0
$graph:
- class: Workflow
  ...

- class: CommandLineTool
  id: composite-cl
  requirements:
    DockerRequirement:
      dockerPull: docker.io/terradue/composite-container
    InlineJavascriptRequirement: {}
  baseCommand: composite
  arguments:
  - $( inputs.tifs[0].path )
  - $( inputs.tifs[1].path )
  - $( inputs.tifs[2].path )
  inputs:
    tifs:
      type: File[]
    lineage:
      type: Directory
      inputBinding:
        position: 4
  outputs:
    rgb_composite:
      outputBinding:
        glob: .
      type: Directory
...

Adding this tool to a new workflow of two command-line tools it is possible to define an application that accepts a Sentinel-2 product, selects the bands, crops them and creates a composite. Below is the interface for the application with input product, the red, green and blue bands together with the bounding box and projection.

cwlVersion: v1.0
$graph:
- class: Workflow
  label: Sentinel-2 RGB composite
  doc: This application generates a Sentinel-2 RGB composite over an area of interest
  id: s2-compositer
  inputs:
    product:
      type: Directory
      label: Sentinel-2 inputs
      doc: Sentinel-2 Level-1C or Level-2A input reference
    red:
      type: string
      label: red channel
      doc: Sentinel-2 band for red channel
    green:
      type: string
      label: green channel
      doc: Sentinel-2 band for green channel
    blue:
      type: string
      label: blue channel
      doc: Sentinel-2 band for blue channel
    bbox:
      type: string
      label: bounding box
      doc: Area of interest expressed as a bounding box
    proj:
      type: string
      label: EPSG code
      doc: Projection EPSG code for the bounding box coordinates
      default: "EPSG:4326"
  outputs:
    ...
- class: CommandLineTool
  id: crop-cl
  ...
- class: CommandLineTool
  id: composite-cl
  ...
...

The workflow orchestration is managed by the steps field where the crop tool extracts an area of a Sentinel-2 product then the composite tool that creates an image of the product with the selection of three bands for the red, green and blue channels.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-compositer
  ...
  inputs:
    ...
  outputs:
    results:
      outputSource:
      - node_composite/rgb_composite
      type: Directory
  steps:
    node_crop:
      run: "#crop-cl"
      in:
        product: product
        band: [red, green, blue]
        bbox: bbox
        epsg: proj
      out:
        - cropped_tif
      scatter: band
      scatterMethod: dotproduct
    node_composite:
      run: "#composite-cl"
      in:
        tifs:
          source:  node_crop/cropped_tif
        lineage: product
      out:
        - rgb_composite

- class: CommandLineTool
  id: crop-cl
  ...
  outputs:
    cropped_tif:
      outputBinding:
        glob: '*.tif'
      type: File

- class: CommandLineTool
  id: composite-cl
  ...
  outputs:
    rgb_composite:
      outputBinding:
        glob: .
      type: Directory

...

This workflow can be visualized as shown in the next figure.

Figure 7 — Workflow diagram for the Application with six inputs parameters and two execution blocks

To execute the previous workflow with multiple input EO products it is only necessary to create a parent workflow that will replicate the input parameters and an array of input products. Together with the scatter requirements this workflow will process multiple composite images.

This workflow below takes a list of products as input and invokes a two-step sub-workflow that crops (using scatter over the bands) and creates a composite.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-composites
  label: Sentinel-2 RGB composites
  doc: This application generates a Sentinel-2 RGB composite over an area of interest with selected bands
  requirements:
  - class: SubworkflowFeatureRequirement
  - class: ScatterFeatureRequirement
  inputs:
    products:
      type: Directory[]
      label: Sentinel-2 inputs
      doc: Sentinel-2 Level-1C or Level-2A input references
    red:
      type: string
      label: red channel
      doc: Sentinel-2 band for red channel
    green:
      type: string
      label: green channel
      doc: Sentinel-2 band for green channel
    blue:
      type: string
      label: blue channel
      doc: Sentinel-2 band for blue channel
    bbox:
      type: string
      label: bounding box
      doc: Area of interest expressed as a bounding bbox
    proj:
      type: string
      label: EPSG code
      doc: Projection EPSG code for the bounding box coordinates
      default: "EPSG:4326"

  outputs:
    wf_results:
      outputSource:
      - node_rgb/results
      type: Directory[]

  steps:
    node_rgb:
      run: "#s2-compositer"
      in:
        product: products
        red: red
        green: green
        blue: blue
        bbox: bbox
        proj: proj
      out:
      - results
      scatter: product
      scatterMethod: dotproduct

- class: Workflow
  id: s2-compositer
  label: Sentinel-2 RGB composite
  doc: This sub-workflow generates a Sentinel-2 RGB composite over an area of interest

- class: CommandLineTool
  id: crop-cl
  ...
- class: CommandLineTool
  id: composite-cl
  ...

The full Application Package file is included in Annex D.

8.7.  Application Additional Metadata

The Application Package can include additional metadata in CWL descriptions and developers should provide a minimal amount of authorship information to encourage correct citation.

It is recommended to include additional metadata in the Application Package using schema.org class Person to define the author and contributions and properties like citation, codeRepository, dateCreated and license as seen in the next example.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...
- class: CommandLineTool
  id: crop-cl
  ...
$namespaces:
  s: https://schema.org/
s:softwareVersion: 1.0.0
schemas:
- http://schema.org/version/9.0/schemaorg-current-http.rdf

It is recommended that concepts from schema.org are used whenever possible and linked with their RDF encoding. Table 1 lists the selected elements recommended by the Best Practice for Earth Observation Application Package.

Table 1 — Application Package additional Metadata elements

8.8.  Resources for the runtime environment

CWL provides a mechanism for expressing runtime environment resource requirements with the simple rule:

• min is the minimum amount of a resource that must be reserved to schedule a job. If min cannot be satisfied, the job should not be run.

• max is the maximum amount of a resource that the job shall be permitted to use. If a node has sufficient resources, multiple jobs may be scheduled on a single node provided each job’s “max” resource requirements are met. If a job attempts to exceed its “max” resource allocation, an implementation may deny additional resources, which may result in job failure.

Hardware resources are expressed with the CWL “ResourceRequirement” allowing the definition of:

• coresMin for the minimum reserved number of CPU cores

• coresMax for the maximum reserved number of CPU cores

• ramMin for the minimum reserved RAM in mebibytes

• ramMax for the maximum reserved RAM in mebibytes

This definition covers most of the application resource requirements needs.​

If appropriate the Application Package can define resources for the runtime environment with ResourceRequirement class either at the level of each CommandLineTool classes or at the level of the Workflow class (that will be inherited to all CommandLineTool classes)

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...
  requirements:
    ResourceRequirement:
      ramMin: 10240
      coresMin: 3

- class: CommandLineTool
  id: crop-cl
  ...

8.9.  Requirement Classes

9.1.  Overview

A Platform that complies with the Best Practice for Earth Observation Application Package needs to provide a mechanism to deploy the Application Package and a mechanism to execute the process defined by the Application Package (i.e. create a new job) with specific parameters.

For the deployment, the platform needs to:

• Accept a Post request with an Application Package (OGC API — Processes)

• Translate Application Package metadata to create a new process in the OGC API — Processes instance

• Translate Application Package Workflow Inputs defined in the CWL document as OGC API — Processes parameters

• Create a new Process offering in the OGC API — Processes

Figure 8 — Platform steps for the EO Application Package deployment

For the execution, the platform needs to:

• Translate OGC API — Processes execute parameters to the Workflow Inputs defined in the Application Package (CWL document)

• If applicable, execute the data stage-in for the input EO products

• Orchestrate and execute CWL

• Translate output to OGC API process outputs

Figure 9 — Platform steps for the EO Application Package execution

A possible diagram of such a platform is shown in the next figure where the Platform presents an OGC API Processes interface for a CWL Conformant Executor that performs the service request in a processing cluster.

Figure 10 — High level diagram of Platform Components executing an EO Application Package

9.2.  Process Description

The Application Package defined in the previous section provides the document that formally defines the inputs, outputs and other necessary metadata about a process that is to be deployed through the API.

For the Application Package, the Workflow class level is the interface used to map the parameters of the OGC API — Processes Web Service and respective mapping with the exposed service. The CWL Workflow class defines the overall processing service and includes two main sections: one for Service definition and the other for Service parameters.

As shown in the next example there is a main section for the Workflow class (extended with the schema.org classes) and two additional sections for the inputs and outputs. These fields are used to comply with the required information for an OGC process description.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  label: This application crops a Sentinel-2 band
  doc: This application crops a band from a Copernicus Sentinel-2 product using GDAL
  ...
  inputs:
    ...
  outputs:
  ...
s:version: 1.0
s:keywords: Sentinel-2, Copernicus

$namespaces:
 s: https://schema.org/
$schemas:
 - http://schema.org/version/latest/schemaorg-current-http.rdf

The Application Package properties defined in CWL are mapped according to the process description of the OGC API Processes as shown below.

{
  "id": "s2-cropper",
  "title": "This application crops a Sentinel-2 band",
  "description": "This application crops a band from a Copernicus Sentinel-2 product using GDAL",
  "keywords": [ "Sentinel-2", "Copernicus" ],
  "inputs": [
    ...
  ],
  "outputs": [
    ...
  ],
  "version": 1.1,
  "jobControlOptions": [ "async-execute"],
  "outputTransmission": [ "reference" ],
  "links": [
    ..
  ]
}

The Workflow fields id, label and doc describe respectively the Process Description elements id, title and description as shown in Table 2.

Table 2 — Mapping the Workflow class fields to OGC API Processes

9.3.  Input Parameters

​ The input parameters of the application package are defined on the inputs section and are used to map the input parameters defined in the CWL to the OGC process description as used in the OGC API — Processes implementation. The process description language may use JSON Schema fragments to define the input and output parameters of a process.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: s2-cropper
  ...
  inputs:
    ...
    band:
      type: string
      label: Sentinel-2 band to crop
      doc: Sentinel-2 band to crop (e.g. B02)
  ...

The workflow call properties are mapped accordingly to the process description of the OGC API Processes as shown below.

{
  "id": "s2-cropper",
  ...
    "inputs": {
      ...
      "band": {
        "title": "Sentinel-2 band to crop ",
        "description": "Sentinel-2 band to crop (e.g. B02)"
        "schema": {
          "type": "string"
        }
      },
      ...
    },
  ...
  "outputs": {
    ...
  },
  ...
}

The following sections describe the rules for mapping the input parameters.

cwlVersion: v1.0
$graph:
- class: Workflow
  id: any-service
  ...
  inputs:
    ...
    my_parameter:
      label: Title of integer parameter
      doc: Abstract describing integer parameter
      type: int?
      default: 100
  ...

{
  ...
    "inputs": {
      ...
      "my_parameter": {
        "title": "Title of integer parameter  ",
        "description": "Abstract describing integer parameter"
        "schema": {
          "type": "integer"
          "nullable": "true"
        }
      },
      ...
    }
  ...
}

cwlVersion: v1.0
$graph:
- class: Workflow
  id: any-service
  ...
  inputs:
    ...
    my_array_parameter:
      label: Title of integer array parameter
      doc: Abstract describing integer array parameter
      type: int[]
  ...

{
  ...
    "inputs": {
      ...
      "my_array_parameter": {
      "title": "Title of integer array parameter",
      "description": "Abstract describing integer array parameter",
      "schema": {
        "type": "array",
        "minItems": 1,
        "maxItems": unbounded,
        "items": {
          "type": "integer"
          }
        }
      },
      ...
    }
  ...
}

cwlVersion: v1.0
$graph:
- class: Workflow
  id: any-service
  ...
  inputs:
    ...
    string_with_options_parameter:
      label: Title of string_with_options_parameter
      doc: This parameter accepts a list of possible values
      type:
        type: enum
        Symbols: [‘option1’, ‘options2’, ‘option3’]
  ...

{
  ...
    "inputs": {
      ...
      "string_with_options_parameter": {
        "title": "Title of string_with_options_parameter",
        "description": "This parameter accepts a list of possible values",
        "schema": {
          "type": "string",
            "enum": [
              "option1",
              "option2",
              "option3"
            ]
        }
      },
      ...
    }
  ...
}

cwlVersion: v1.0
$graph:
- class: Workflow
  id: any-service
  ...
  inputs:
    ...
    aux_file:
      label: Auxiliary File
      doc: This is an auxiliary file needed for the processing
      type: File
      format: text/xml
  ...

 {
  ...
    "inputs": {
      ...
      "aux_file" : {
        "title": "Auxiliary File",
        "description": "This is an auxiliary file needed for the processing",
        "minOccurs": "1", "maxOccurs": "1",
        "schema": {
          "type" : "string",
          "contentMediaType" : "text/xml"
        }
      },
      ...
    }
  ...
 }

{
  ...
  "inputs": {
    ...
    "input_reference" : {
      ...
      "formats": [
        {
          "mimeType": "application/json",
          "schema" : "https://raw.githubusercontent.com/stac-extensions/single-file-stac/main/json-schema/schema.json"
          "default": true
        },
        {
          "mimeType": "application/geo+json",
          "schema" : "https://raw.githubusercontent.com/stac-extensions/single-file-stac/main/json-schema/schema.json"
        }
      ]
   ...
    }
  ...
  }
}

9.4.  Data Flow Management

The Platform is responsible for the data flow management by using a local catalogue encoded using the SpatioTemporal Asset Catalog (STAC) specification as a data manifest for application inputs and outputs. The local catalogue provides knowledge about the input and output files data contents like spatial footprint, sub-items (e.g. masks, bands) and additional metadata.

This section describes the strategy to Data stage-in, locally retrieving the inputs products for the processing, and Data stage-out, making the outputs of the processing available for the subsequent steps (locally or on external systems).

9.4.1.  Data Stage-In

The processing inputs are provided as EO Catalogue references and the Platform is responsible for translating those references into inputs available for the local processing.

For each File or Directory parameter, the platform resolves the resources, stages the data for processing. The full steps of the Platform for data stage-in are described in the Figure below.

Figure 11 — Platform steps for data stage-in from the CWL service definition and OGC API Processes execution request

9.4.2.  Data Stage-Out

The data stage-out is the Platform operation of retrieving the application execution results and pushing them to a persistent storage.

The application package uses CWL statements to define what results are pushed from the execution container to the Platform local file system. A typical CWL execution engine also provides a simple manifest with the list of generated results.

The application execution provides a local STAC Catalog including items and assets describing the results generated. This local STAC Catalog is the application results manifest.

The Platform takes the STAC Catalog (catalog.json) file as the generated results entry point and uses the href links to items and assets and uses the found metadata and physical files to push them to a persistent storage and/or catalog and finally provide the OGC API Processes execution response.

The diagram below shows how the Platform does the steps explained above.

Figure 12 — Platform steps for data stage-out application execution for the the OGC API Processes response

9.5.  Requirement Classes