OGC Testbed-16: Machine Learning Engineering Report

The OGC API framework is organized by resource type (e.g. features, coverages, maps, tiles). Each resource has an associated API standard. These resource-specific API standards are built using shared API modules. The OGC API-Common suite of standards stages the requirements and conformance that define these shared modules.

OGC API - Common - Part 1: Core defines the resources and operations which are be common to all OGC API standards. This draft Standard defines the minimal requirements for an API to be discovered and used by any client.

OGC API - Common - Part 2: Collections provides a common connection between the API landing page and resource-specific details for collections. That connection includes metadata which describes the collections of hosted resources, common parameters for selecting subsets of those collections, and URI templates for identifying the above.

The OGC API - Features Standard defines API building blocks to create, modify and query features on the Web. OGC API - Features is comprised of multiple parts, each of which is a separate standard.

OGC API - Features - Part 1: Core specifies the core capabilities and is restricted to fetching features where geometries are represented in the World Geodetic System 1984 coordinate reference system (WGS 84) with axis order longitude/latitude. The API further specifies discovery and query operations that are implemented using the HTTP GET method.

By default, every API implementing Part 1 will provide access to a single dataset. Rather than sharing the data as a complete dataset, OGC API Features offers direct, fine-grained access to the data at the feature (object) level.

Discovery operations defined in Part 1 enable clients to interrogate the API to determine its capabilities and retrieve information about this distribution of the dataset, including the API definition and metadata about the feature collections provided by the API.

Query operations defined in Part 1 enable clients to retrieve features from the underlying data store based upon simple selection criteria, defined by the client.

OGC API - Features - Part 2: Coordinate Reference Systems by Reference extends Part 1 to support additional coordinate reference systems in addition to WGS84 CRS specified in the core. This is an approved OGC standard.

OGC API - Features - Part 3: Filtering and the Common Query Language (CQL) extends Part 1 to support richer filtering capabilities beyond the simple selection capabilities defined in Part 1. This is a draft standard.

The API building blocks specified in the OGC API - Features suite of standards are consistent with the architecture of the Web. In particular, the API design is guided by the Internet Engineering Task Force (IETF) HTTP/https://tools.ietf.org/html/rfc2818[HTTPS] RFCs, the W3C Data on the Web Best Practices, the W3C/OGC Spatial Data on the Web Best Practices and the emerging OGC Web API Guidelines. A particular example is the use of the concepts of datasets and dataset distributions as defined in the W3C Data Catalog Vocabulary (DCAT) and used in schema.org.

In this thread, a features server was deployed to offer vector data for the training of ML models and as input data when executing trained models. Specifically, an OGC API Features server supporting both parts 2 and 3 was deployed to host the Canvec Series of Topographic Data of Canada.

Coverages - as per OGC, and ISO standardization - constitute a unifying paradigm for the digital representations of space/time varying phenomena. Specifically, these are data defined as spatio-temporal regular and irregular grids, point clouds, and general meshes. In particular, multi-dimensional datacubes fall under this category, such as 1-D sensor time series, 2-D satellite imagery, 3-D x/y/t image time series and x/y/z geoscientific models, 4-D x/y/z/t climate and ocean data sets, and more.

The draft OGC API - Coverages standard establishes how to access coverages. Like OGC API Features, OGC API - Coverages is comprised of multiple parts, each of them as a separate standard.

The draft OGC API - Coverages - Part 1: Core, established how to access coverages as defined by the Coverage Implementation Schema (CIS) 1.1.

Part 1 defines coverage extraction and includes subsetting capabilities, including band subsetting, scaling, CRS conversion and data format encoding. Gridded coverages or 'point-cloud' multi-point coverages are the primary data types considered but other kinds of datasets are not excluded.

In this Testbed thread, a coverage server was deployed serving Sentinel S1A and S1B satellite imagery from Canada. This data was used for training ML models and as input when executing trained models.

¹: The expression "geospatial resources" means "from more than one geospatial resource or collection" ²: Specified in the draft OGC API - Maps – Part 1: Core specification.

OGC API - Processes - Part 1: Core, specifies building blocks that enable the execution of geospatial computing processes on the Web. The draft standard defines requirements and conformance for providing process inputs and for retrieving the results of processing. Processes can be executed synchronously or asynchronously. The API also defines discovery resources for the retrieval of metadata describing the purpose and functionality of each process. Typically, these processes combine raster, vector, coverage and/or point cloud data with well-defined algorithms to produce new raster, vector, coverage and/or point could information.

Part 1 of OGC API - Processes only offers a static set of processes. Previous OGC Testbed work in OGC Testbeds 13 and 14 extended the processes API to allow for the dynamic deployment of processes. This extension is currently referred to as the Application Deployment and Execution Service.

The draft OGC API - Processes - Part 2: Transactions specification formalizes the work done in testbeds 13, 14 and 15 to extend the core to add transactional capability; that is the ability to dynamically deploy and "undeploy" processes.

A service that implements these two parts of the OGC API - Processes suite of standards is referred to as an "Application Deployment and Execution Service".

The ADES allows clients to deploy processes bundled in Docker containers and manages the environment required to execute those containers. Typically, this requires that an ADES understand:

The first item requires that the ADES understands how to access data and stage it for use by an application in a Docker container. This may require that the ADES know how to retrieve data from an OGC API such as OGC API - Features or it may require that the ADES know how to read data from other APIs such as Amazon’s S3 object store.

The second item requires that the ADES understand how to interact with the specific platform upon which it is deployed in order to stage data for the Dockerized application and execute the Docker container. For example, an ADES deployed to an Amazon Cloud needs to understand how to interact with AWS’s Elastic Container Service (ECS) or Elastic Kubernetes Service (EKS) in order to execute Docker containers at scale. Similar statements are applicable for ADES' deployed on the Google Cloud Platform or on OpenStack-based cloud such as the Boreal Cloud of Natural Resources Canada (NRCan).

The Execution Management Service (EMS) is basically an ADES that does not actually "execute" processes but rather acts as a coordinating node in a federation of ADESs. Typically, an EMS knows how to:

So, even though the API presented by the EMS is the same as that presented by the ADES the role of each component is very different. It should be noted, however, that many implementations can be deployed for fulfill either or both roles (i.e. EMS or ADES).

The following API summary also includes elements of the OGC API - Processes transactional extension which allows processes to be dynamically deployed and undeployed thus extending the API.

OGC API - Records - Part 1: Core defines an API that supports the ability to search collections of descriptive information, called records, about resources such as data collections, services, processes, styles, code lists and other related resources. Records represent resource characteristics that can be queried and presented for evaluation and further processing by both humans and software.

Part 1 defines a set of core queryables that represent a list of mandatory and optional properties that a catalogue records should contain to provide a minimum, useful amount of information about the resource the record is describing.

The draft API for Part 1 is very similar to that for OGC API - Features - Part 1: Core but also includes additional query parameters that extend the query capabilities of the API.

Part 1 also makes recommendations about the encoding of a record. GeoJSON is specified as the JSON encoding of a record. There are recommendations for XML and HTML encoding but these where not used in the ML Testbed activity.

The following table lists mandatory (indicated by "M") and recommended optional ("O") properties that every catalogue record should include. This list of properties is referred to as the "core queryables".

The following table lists the set of query parameters that every implementation of the OGC API - Records draft standard must provide. The bbox, datetime and limit parameters are inherited from OGC API - Features - Part 1: Core standard.

Early in the preparation phase of the model development, different data sources were taken into consideration. In particular, the Radarsat-1 data provided by NRCan was assessed with regards to its feasibility within the ML application. For pre-processing the Sentinel Application Platform (SNAP) built-in Radarsat importer was used. A set of issues arose in the course of this workflow:

After consideration with SAR experts at NRCan participants decided to opt for a Sentinel-1 based model approach as the data was very well supported in terms of pre-processing functionality and accuracy.

Two training datasets where used: pre-processed Sentinel-1 scenes and water body data (the CanVec dataset "Lakes, Rivers and Glaciers in Canada - CanVec Series - Hydrographic Features"; see https://open.canada.ca/data/en/dataset/9d96e8c9-22fe-4ad2-b5e8-94a6991b744b). As described in section OGC API - Maps and OGC API - Tiles (Drafts), both the Sentinel-1 and the label data were provided by a prototype OGC API Tiles implementation. The below Figure illustrates the tile-based approach.