3.1.  Terms and definitions

3.1.1. Application Programming Interface (API)

an interface that is defined in terms of a set of functions and procedures and enables a program to gain access to facilities within an application [8].

3.1.2. Façade Service

a component that fetches data from a specific data source and makes it available through its own interface [10]. The main reason for building this type of service is the difficulty or inability to modify the original data source with the intent of modifying:

• the underlying structure of the API; and

• the format in which the data are made available.

3.1.3. filter expression

predicate encoded for transmission between systems.

Example

[SOURCE: [14]]

3.1.4. parameterized stored query

a stored query that has one or more parameters.

3.1.5. Predicate

set of computational operations applied to a data instance which evaluate to true or false.

[SOURCE: [5]]

3.1.6. property selection

operation to create a copy of a data instance, restricted to a subset of the properties of the data instance.

3.1.7. query

request for data from a dataset.

3.1.8. queryable

a token that represents a property of a resource that can be used in a filter expression.

[SOURCE: [14]]

3.3.  sortable

a token that represents a property of a resource that can be used to sort a collection of resource instances.

3.3.1. sorting

operation to order the data instances in a set based on the values of selected properties of each data instance.

3.3.2. Standardized API

an API that is intended to be deployed by multiple API providers with the same API definition.

3.3.3. Standards-based API

an API that conforms to one or more conformance classes specified in one or more standards.

3.3.4. stored query

a predefined query that is available a resource in a Web API.

3.3.5. SWIM Data

any data provided through the SWIM System.

3.3.6. Web API

an API using an architectural style that is founded on the technologies of the Web [9].

3.4.  Abbreviated terms

AIXM

Aeronautical Information Exchange Model

API

Application Programming Interface

CQL2

OGC Common Query Language

CRS

Coordinate Reference System

ER

Engineering Report

FAA

Federal Aviation Administration

JSON

JavaScript Object Notation

NAS

National Airspace System

NOTAM

Notice to Airmen

OGC

Open Geospatial Consortium

SFDPS

SWIM Flight Data Publication Service

SWIM

System Wide Information Management

TB

Testbed

TIE

Technology Integration Experiment

WFS

Web Feature Service

4.1.  Background

4.1.3.  Exploration of OGC API Standards by SWIM

Over the years, the FAA and the OGC have jointly explored making SWIM data more easily accessible and more valuable. As part of these past efforts, Testbed-16 brought together previous work on the development of OGC APIs, the use of semantics to enrich data, and SWIM data processing. The objectives were to deliver the first demonstration of an OpenAPI-based API serving SWIM data, a component generating aviation Linked Data, and two client applications querying and displaying that data [15].

Two of TB-16 recommendations were to integrate OGC API requirement classes within SWIM Data Services and to demonstrate interoperability between diverse Aviation APIs [15]. In order to advance these recommendations, TB-17 focused on the development of eleven APIs based on OGC API Standards and the completion of Technology Integration Experiments (TIEs) between these APIs.

During TB-16, the development of the API serving aviation data resulted in numerous lessons learned and recommendations [15]. TB-16 saw the development of one aviation-related API based on an OGC API Standard (OGC API — Features). The APIs developed during TB-17 ([4]) addressed many of those lessons learned and implemented additional OGC API Standards (draft and approved) which have been maturing since. This process is reflected in Figure 2.

Figure 2 — History of OGC experiments to enhance SWIM

4.2.  Requirements Statement

Testbed-18 required investigating the potential of filtering using OGC API Standards in the context of the SWIM Program.

The original goals of the TB-18 Advanced Filtering of SWIM Feature Data Task were as follows.

• Experiment with OGC API-Features filtering mechanisms.

• Explore if best practices for advertising filtering capabilities are required beyond what is already defined in the various OGC API-Features Parts.

• Demonstrate advanced filtering in situations where the data endpoints support only rudimentary filtering by introducing a new service type “Filtering Service.”

• Allow filtering rules for a specific data service to be provided at runtime in a machine-readable manner.

The research questions for the Advanced Filtering of SWIM Feature Data Task were as follows.

• How does filtering of SWIM data served by OGC API-Features endpoints work?

• Is the metadata required by the various OGC API-Features parts sufficient to allow clients to fully “understand” the filtering capabilities of a service endpoint?

• OGC API — Features — Part 3: Filtering and the Common Query Language (CQL) supports queryables that are not directly represented as resource properties in the content schema of the resource. Is it possible to identify best practices for their usage?

• Clients may know the content schema of offered resources. How best to use this knowledge for advanced filtering beyond what is defined in OGC API — Features — Part 3: Filtering and the Common Query Language (CQL)?

• How does a filtering service look that allows advanced filtering for rather simple OGC API-Features-based SWIM data endpoints?

• How does such a service work in situations where a data publisher has restricted filtering on certain properties (for example, because the backend datastore has not been configured to allow high-performance queries on those properties)?

• How can a client application support a customer who has knowledge of the content schema of an offered resource in the creation of filter statements? What are the key requirements for a developer GUI that supports visualization and management of these filtering tools?

• Is it possible to easily create a new filtered dataset by creating machine readable filtering rules based on the metadata required by the OGC API-Features standards? How can these rules be provided to the Filtering Service at runtime?

4.3.  Functional Overview

As shown in Figure 3, the Advanced Filtering of SWIM Feature Data Task architecture was organized into a system of seven interconnected components. All seven components were developed simultaneously throughout the Testbed, with permanent communication and cooperation among participant organizations.

The components can be divided into the following three groups.

• Façades for SWIM services with simple filtering mechanisms. Retrieve aviation data from multiple SWIM services and serve these data through APIs built based on OGC API Standards featuring basic filtering mechanisms. Three Façades were built.

  • The OGC API-Features Façade 1 (identified collectively as D100): Four APIs built to serve NOTAMs, Airport Layouts, and Airspaces

  • The OGC API-Features Façade 2 (identified collectively as D101): Three APIs built to serve aeronautical, flight, and weather features.

  • An extra façade, not originally included in the Task architecture, was offered in-kind by the company Skymantics, and was named OGC API-Features Façade 3: An API built to serve flight plans from the SFDPS (FAA) Service.

• Components that serve aviation data with advanced filtering mechanisms. Two filtering services were built, each one featuring an API.

  • The Filtering Service 1 (identified as D102): Built to serve SWIM data from D100 with advanced filtering mechanisms.

  • The Filtering Service 2 (identified as D103): Built to serve SWIM data from all three façades with advanced filtering mechanisms.

• Client components to demonstrate consumption of filtered data and configuration of filtering mechanisms. Two clients were built: One meant to serve an aviation domain expert and the other to serve a developer of aviation software applications.

  • The Business User Client (identified as D104): A client built to query filtering services and demonstrate the usage of advanced filtering mechanisms.

  • The Developer Client (identified as D105): A client built to define filter statements that can be expressed in a machine-readable way and exchanged with the filtering services.

Figure 3 — Component Diagram for the Advanced Filtering of SWIM Feature Data Task

4.3.1.  Component Interactions

The following two figures illustrate the intended interactions between the components described in the Work Items & Deliverables section of this ER. The two figures illustrate the workflows for using the filtering service for data subsetting (Figure 4) from the perspective of a business client and for configuring the filtering service at runtime (Figure 6) from the perspective of the filtering rules developer.

In the first workflow, illustrated in Figure 4, an OGC API-Features façade to SWIM Data Service data service offers insufficient filtering capabilities to its customers. The Business User Client does not want to access large data sets and then perform filtering itself. Instead, the client wants to make use of a Filtering Service that can handle the filtering of the data and provide the subset of the data that the client is interested in. If the filtering service receives a data request from the client, it connects to the data service to access the necessary data, filters out everything that is not requested by the client, and eventually delivers the result to the client.

Figure 4 — Workflow from the perspective of a business user that needs filtered data

Figure 5 — First Workflow Sequence Diagram

The second workflow, illustrated in Figure 6, demonstrates how a filtering service can be configured at run time. The assumption is that the Developer Client is aware of the API characteristics of the data service as well as the content schema of the data served by the data server. Based on both, the client supports the user with a GUI in the definition of the filtering rules. The user can then register these rules with the filtering service, which is now configured to run the data service specific filtering.

Figure 6 — Workflow from the perspective of a filtering rules developer

Figure 7 — Second Workflow Sequence Diagram

5.1.  Conformance Classes

• Core: Support executing stored queries.

• Parameterized Stored Queries: Support executing parameterized stored queries.

• Manage Stored Queries: Support reading, creating, replacing, and deleting stored queries.

• Ad-hoc Queries: Support executing ad-hoc queries.

Table 2 — Conformance Classes

Note that /search/{queryId}/definition is part of the ‘Manage Stored Queries’ Conformance Class. This is because in the current design GET on /search returns only the main query metadata and, if applicable, the parameter descriptions. This supports use cases where a query provider wants to keep the query expressions hidden from regular users. That is, the query expression of a stored query should only be visible to those managing the query. Consequently, GET on /search/{queryId}/definition is part of the Manage Stored Queries conformance class.

6.1.  Query Expression

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "QueryExpression.json",
  "oneOf": [
    {
      "allOf": [
        {"$ref": "#/$defs/query"},
        {
          "type": "object",
          "properties": {
            "title": {"type": "string"},
            "description": {"type": "string"},
            "limit": {"$ref": "#/$defs/limit"},
            "parameters": {"$ref": "Parameters.json"}
          }
        }
      ]
    },
    {
      "type": "object",
      "required": ["queries"],
      "properties": {
        "title": {"type": "string"},
        "description": {"type": "string"},
        "queries": { "type": "array", "minItems": 1, "items": {"$ref": "#/$defs/query"} },
        "filter": {"$ref": "#/$defs/filter"},
        "filterOperator": { "type": "string", "enum": ["and", "or"], "default": "and" },
        "properties": {"$ref": "#/$defs/properties"},
        "limit": {"$ref": "#/$defs/limit"},
        "parameters": {"$ref": "Parameters.json"}
      }
    }
  ],
  "$defs": {
    "query": {
      "type": "object",
      "required": ["collections"],
      "properties": {
        "collections": { "type": "array", "minItems": 1, "items": {"type": "string"} },
        "filter": {"$ref": "#/$defs/filter"},
        "properties": {"$ref": "#/$defs/properties"},
        "sortby": {"$ref": "#/$defs/sortby"}
      }
    },
    "filter": {"type": "object"},
    "properties": { "type": "array", "minItems": 1, "items": {"type": "string", "minLength": 1} },
    "sortby": {
      "type": "array",
      "minItems": 1,
      "items": {"type": "string", "pattern": "[+|-]?.+"}
    },
    "limit": {"type": "integer", "minimum": 1, "default": 1000, "maximum": 10000}
  }
}

A query expression can contain a single query (the properties “collections,” “filter,” “properties,” and “sortby” are members of the query expression object) or multiple queries (a “queries” member with an array of query objects is present) in a single request.

For each query

• The value of “filter” is a CQL2 JSON filter expression.

• The value of “properties” is an array with the names of properties to include in the response.

• The value of “sortby” is used to sort the features in the response.

• Multiple entries in the “collections” member represent a join between the specified collections. Just like in SQL, the properties of each collection participating in the join are combined and presented in the features in the result set. Property names in the request and in the response have to be prefixed with the collection name plus a period (“.”), e.g., “collection.property.”

  • Support for joins was a stretch goal in Testbed 18 and not discussed or tested in detail.

  • There are several open questions related to joins that should be considered by the Features API SWG.

    • How should the id of the joined feature be assigned? A combination of all features in the tuple?

    • How should the primary geometry be assigned (in the GeoJSON representation) if there are multiple geometry properties?

    • Should it also be allowed to “nest” joined properties, so instead of { …​, "col.prop1": 1, "col.prop2": "a", …​ } encode it as { …​, "col": { "prop1": 1, "prop2": "a" }, …​ }?

For multiple queries

• If multiple queries are specified, the results are concatenated. The response is a single feature collection.

  • The feature ids in the response to a multi-collection query must be unique. Since the featureId of a feature only has to be unique per collection, they need to be combined with the collectionId. The server could determine how the id values are constructed or require a specific approach. The latter has the advantage that the source feature could be identified (if, e.g., a concatenation of the collectionId and featureId such as “apronelement.123456” is used). However, if this is a requirement, then maybe a “self” link in each feature would be cleaner? In addition, if an API already uses feature identifiers that are unique across all collections (e.g., a UUID), then a mandatory collection prefix would be unnecessary.

  • Another aspect is cases where a feature is included in the result set of multiple queries. The feature should be included only once in the result. However, if the ids are constructed in a way that there is no conflict, if the same feature is included multiple times (e.g., just an auto-incrementing index value as the id), maybe it could also be tolerated if the same feature is included more than once?

  • Another approach to both issues could be to return an array of feature collections in the response, one for each query. Similar to what was done in WFS 2.0. This seems cleaner and clearer as it avoids the “hacks” and issues discussed in the previous bullet items. Since /search is a different resource than /items this would be conceptually clean. This leaves the question of usability: a single feature collection is easier if the result is just fed into some GeoJSON tooling. However, in a JavaScript based client it seems there is no overhead to process such a response. It could also be beneficial for use with JSON-FG where each feature collection would more likely be homogeneous and could include metadata that simplify parsing the response.

• The direct members “filter” and “properties” represent “global” constraints that must be combined with the corresponding member in each query. The global and local property selection list are concatenated and then the global and local filters are combined using the logical operator specified by the “filterOperator” member.

  • The global member “filter” should only reference queryables that are common to all collections being queried. If a queryable is specified that is not defined or does not exist for a particular collection then the value of the property is null.

  • The global member “properties” should only reference presentables that are common to all collections being queried. If a presentable is specified that is not defined or does not exist for a particular collection then the property is omitted from the response.

General rules

• A “title” and “description” for the query expression can be added. Providing both is strongly recommended to explain the query to users.

• The “limit” member applies to the entire result set.

• Note that “sortby” will only apply per query. A global “sortby” would require that the results of all queries are compiled first and then the combined result set is sorted. This would not support “streaming” the response.

• In case of a parameterized stored query, the query expression may contain JSON objects with a member “$parameter.” The value of “$parameter” is an object with a member where the key is the parameter name and the value is a JSON schema describing the parameter. When executing the stored query, all objects with a “$parameter” member are replaced with the value of the parameter for this query execution. Comma-separated parameter values are converted to an array if the parameter is of type “array”.

• Parameters may also be provided in a top-level member “parameters” and referenced using “$ref”.

6.2.  Stored Queries

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "StoredQueries.json"
  "type": "object",
  "properties": {
    "queries"    : { "type": "array" , "items": {"$ref": "#/$defs/StoredQueryDescription"} },
    "links"      : { "type": "array" , "items": {"$ref": "#/$defs/Link"}                   },
    "title"      : { "type": "string"                                                      },
    "description": { "type": "string"                                                      }
  },
  "$defs": {
    "StoredQueryDescription": {
      "required": ["id"],
      "type": "object",
      "properties": {
        "id": {"type": "string"},
        "title": {"type": "string"},
        "description": {"type": "string"},
        "parameters": { "$ref": "Parameters.json" },
        "links": { "type": "array", "items": {"$ref": "#/$defs/Link"} }
      }
    },
    "Link": {
      "type": "object",
      "required": ["href", "rel"],
      "properties": {
        "href": {"type": "string"},
        "rel": {"type": "string"},
        "title": {"type": "string"},
        "type": {"type": "string"},
        "hreflang": {"type": "string"},
        "length": {"type": "integer"},
        "templated": {"type": "boolean", "default": false}
      }
    }
  }
}

6.3.  Parameters

The parameters are described as a JSON object where each parameter is a property with its JSON Schema as its value.

Providing sufficient information that allows clients to generate meaningful queries is essential. Recommendation 1 in OGC API — Features — Part 3 has recommendations for schemas that are straightforward to parse by clients but that are expressive enough to allow clients to generate forms to provide parameter values.

In addition, if a parameter declares a default value, the API will use that default value if no value is provided in the request to execute a query.

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "Parameters.json"
  "type": "object",
  "description": "Each parameter is described by a property where each value is a JSON Schema object.",
  "additionalProperties": {
    "$ref": "https://json-schema.org/draft/2019-09/schema"
  }
}

7.1.  Pre-defining queries based on Processes — Part 3 extension

The OGC API — Features Search extension (described in detail in the other section of this document) shares a lot in common with the idea of allowing filter, properties (for selection and derived fields/properties), and sortBy to qualify inputs in OGC API — Processes — Part 3: Workflows and Chaining (input and output modifiers requirements classes).

A well-known pass-through process (with support for collection input including filtering) could support an execution request with a syntax equivalent to the Search extension endpoint, similar to how the OGC API — Routes /routes endpoint shares a POST payload syntax with an eventual definition of a well-known routing process. Pre-defined queries could also be parameterized by deploying them as processes, as suggested in the Deployable workflows requirements class of Processes — Part 3: Workflows and Chaining.

The modifiers introduced include the same filter, properties, and sortBy parameters to qualify inputs originating from a data collection or process, whether they are local or remote, as well as outputs resulting from a process. In addition to the ability to select specific fields/properties, the properties parameter can also be used to derive new fields, for example using CQL2 arithmetic expressions.

Processes — Part 3 also defines a Collection output requirements class where the output of the workflow execution is either a dataset landing page (which can contain multiple collections), or a single collection.

For example, the following parameterized query, addressing similar use case as the one described further in section 8.3.5 — Support for the Search resources‘s Example 2, taking two parameters, a string enumeration named composition and a string array named airports, could be expressed in a Part 3-extended OGC API — Processes execution request (using Collection input and Output modifiers) as shown below.

Example 1 — Example parameterizable query as a Part 3-extended execution request

{
 "process" : "PassThrough",
 "inputs" : {
  "data" : [ {
    "collection" : "apronelement",
    "filter": {
      "op": "and",
      "args": [
        {
          "op": "=",
          "args": [
            {"property": "composition"},
            {
              "$input": {
                "composition": { "type": "string", "enum": ["CONC", "..."] }
              }
            }
          ]
        },
        {
          "op": "in",
          "args": [
            {"property": "airport"},
            {
              "$input": {
                "airports": {
                  "type": "array",
                  "items": { "type": "string", "enum": ["JFK", "EWR", "LGA", "..."] }
                }
              }
            }
          ]
        }
      ]
    },
    "properties": ["geometry", "airport", "type"],
    "sortBy": ["airport"]
  } ]
 }
}

In this example, data is an input defined in the PassThrough well-known process with multiplicity 1..* which is returned as the process output.

The collection property is defined in the Collection input requirements class of OGC API — Processes — Part 3, whereas the filter, properties, and sortby elements specifying a cql2-json filtering expression, selected properties and an ascending sort order by airport, are defined in Part 3’s Input modifiers requirements class.

These field modifiers can also be used in the context of the Output modifiers requirements class together with the Collection output requirement class or with regular process execution outputs. In the context of a feature collection output, these query parameters building blocks also correspond to the functionality provided by OGC API — Features — Part 3: Filtering, as well as a planned extensions for Coverages and Discrete Global Grid Systems.

This execution request could be deployed as a new process, e.g., ApronFiltering, using the Processes — Part 2: Deploy, Replace, Undeploy extension’s POST operation to /processes together with the Deployable workflows requirements class of Part 3. The resulting process would get listed at /processes with a process description including the input parameters (composition and airports) and could itself be executed by POSTing to /processes/ApronFiltering/execution.

Example 2 — Example execution request of parameterized query deployed as a process

{
  "inputs" : {
    "composition" : "CONC",
    "airports": [ "JFK", "LGA" ]
  }
}

Posting this execution request to the execution endpoint without a Prefer: header would result in a synchronous execution that returns the features. With support for the Collection output requirements class, specifying as parameter response=collection would instead return a collection description, as for a GET request to /collections/{collectionId} in Common — Part 2 and Features — Part 1. With response=landingPage, a landing page would be returned for the filtered dataset, allowing to retrieve multiple collections.

Such virtual collections could also be published to a dataset API as a persistent collection, e.g., as /collections/concApronsJFKLGA, using a non-parameterizable execution request as the payload of a POST operation to /collections to create the dynamic collection, with a Processes execution request content media type to be registered, e.g., application/ogcexecreq+json as suggested in Part 3 — Section 14. Media Types.

During this Testbed-18 advanced filtering task Ecere successfully demonstrated the use of Part 3 extensions to pre-define filtering queries to the cascading service, including the deployment of a sample PassThrough process with support for the filter and properties modifiers, as well as for the Collection Input and Collection output requirements class, with the following limitations:

• the filters were expressed using the cql2-text encoding rather than cql2-json;

• the input and output were limited to a single collection;

• the sortBy modifier (and sorted feature collections in general) remained to be implemented; and

• support for parameterized queries and deployable workflows was not yet implemented.

A sample pre-defined query is available from the endpoint:

https://maps.gnosis.earth/ogcapi/processes/PassThrough/execution?response=collection

including the following default pre-defined filter execution request:

Example 3 — Working execution request of filtering query from D100 cascaded collection

{
   "process" : "https://maps.gnosis.earth/ogcapi/processes/PassThrough",
   "inputs" : {
      "data" : [
         {
            "collection" : "https://maps.gnosis.earth/ogcapi/collections/swim:d100_airports:apronelement",
            "filter" : "composition = 'CONC' and airport in ('JFK', 'EWR', 'LGA')",
            "properties" : [ "geometry", "airport", "type" ]
         }
      ]
   }
}