Published

OGC Engineering Report

Testbed-18: Filtering Service and Rule Set Engineering Report
Sergio Taleisnik Editor
OGC Engineering Report

Published

Document number:22-024r2
Document type:OGC Engineering Report
Document subtype:
Document stage:Published
Document language:English

License Agreement

Use of this document is subject to the license agreement at https://www.ogc.org/license



I.  Abstract

This Testbed-18 (TB-18) Filtering Service and Rule Set Engineering Report (ER) documents best practices identified for features filtering and describes in detail how filtering can be decoupled from data services. Further, this ER describes how filtering rules can be provided to Filtering Services at runtime.

II.  Executive Summary

Previous OGC work addressed the challenges of increasing interoperability between aviation data services. Recently, the OGC community has developed a new family of standardized OpenAPI-based Web APIs for various geospatial resource types. These new OGC API Standards have the potential to enhance the way in which consumers can access geospatial data from various sources. OGC Testbed-16 brought together previous work on the development of OGC API Standards, the use of semantics to enrich data and SWIM data processing, and demonstrated an OpenAPI-based API implementation instance serving SWIM data. OGC Testbed-17 took lessons learned and recommendations from Testbed-16 and focused on further testing the value of standards-based APIs within the SWIM program.

OGC API-Features endpoints define their filtering capabilities. Filtering is standardized across different parts of OGC API-Features (see section Previous Work). As of December 2022, two parts were still in draft status. Advanced filtering capabilities require sophisticated server software. Not all data providers will be able to operate such a powerful service endpoint. FAA SWIM Data Services currently produce data from the National Airspace System (NAS) to consumers using various protocols and service offerings in both synchronous and asynchronous messaging formats. OGC Testbed-18 explored filtering mechanisms for feature data served by OGC API-Features instances. The experiments included filtering of native and fused SWIM data and experimented with filtering services.

The research questions for the Advanced Filtering of SWIM Feature Data Task were as follows.

To answer these questions, this Testbed-18 Task was organized into the development and testing of a system of six interconnected components, as seen on Figure 1.

Figure 1 — Component Diagram for the Advanced Filtering of SWIM Feature Data Task

All components were successfully developed and tested. This ER captures the operations, conformance classes, schemas, and processes meant to support an API with advanced filtering mechanisms. A comprehensive analysis of the research questions is included in a separate chapter.

III.  Keywords

The following are keywords to be used by search engines and document catalogues.

testbed, web service, api, standard, filter, SWIM, aviation


IV.  Preface

It is possible that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

V.  Security considerations

No security considerations have been made for this document.

VI.  Submitters

All questions regarding this document should be directed to the editor or the contributors:

NameOrganizationRole
Sergio TaleisnikSkymantics, LLCEditor
Clemens Porteleinteractive instruments GmbHContributor
Eugene YuGeorge Mason UniversityContributor
Jérôme Jacovella-St-LouisEcere CorporationContributor
Patrick DionEcere CorporationContributor
Mohammad MoallemiConcepts Beyond LLCContributor

Testbed-18: Filtering Service and Rule Set Engineering Report

1.  Scope

This OGC Testbed 18 Engineering Report (ER) documents best practices identified for features filtering and describes in detail how filtering can be decoupled from data services. This includes how filtering rules can be provided to Filtering Services at runtime. The ER specifies operations, schemas, and processes to support search and filtering of features. The ER also documents lessons learned and recommendations for future work.

2.  Normative references

The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.

Open API Initiative: OpenAPI Specification 3.0.2, 2018 https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md

van den Brink, L., Portele, C., Vretanos, P.: OGC 10-100r3, Geography Markup Language (GML) Simple Features Profile, 2012 http://portal.opengeospatial.org/files/?artifact_id=42729

W3C: HTML5, W3C Recommendation, 2019 http://www.w3.org/TR/html5/

Schema.org: http://schema.org/docs/schemas.html

R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee: IETF RFC 2616, Hypertext Transfer Protocol — HTTP/1.1. RFC Publisher (1999). https://www.rfc-editor.org/info/rfc2616.

E. Rescorla: IETF RFC 2818, HTTP Over TLS. RFC Publisher (2000). https://www.rfc-editor.org/info/rfc2818.

G. Klyne, C. Newman: IETF RFC 3339, Date and Time on the Internet: Timestamps. RFC Publisher (2002). https://www.rfc-editor.org/info/rfc3339.

M. Nottingham: IETF RFC 8288, Web Linking. RFC Publisher (2017). https://www.rfc-editor.org/info/rfc8288.

H. Butler, M. Daly, A. Doyle, S. Gillies, S. Hagen, T. Schaub: IETF RFC 7946, The GeoJSON Format. RFC Publisher (2016). https://www.rfc-editor.org/info/rfc7946.

3.  Terms, definitions and abbreviated terms

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

For the purposes of this document, the following additional terms and definitions apply.

3.1.  Terms and definitions

3.1.1. Application Programming Interface (API)

an interface that is defined in terms of a set of functions and procedures and enables a program to gain access to facilities within an application [8].

3.1.2. Façade Service

a component that fetches data from a specific data source and makes it available through its own interface [10]. The main reason for building this type of service is the difficulty or inability to modify the original data source with the intent of modifying:

  • the underlying structure of the API; and

  • the format in which the data are made available.

3.1.3. filter expression

predicate encoded for transmission between systems.

Example

CQL2-Text or CQL2-JSON are examples how a predicate can be encoded as a filter expression.

[SOURCE: [14]]

3.1.4. parameterized stored query

a stored query that has one or more parameters.

Note 1 to entry: When executing a parameterized stored query, the user has to provide parameter values for each parameter of a stored query. If a parameter has a default value, a parameter can be omitted from the request to execute the stored query.

3.1.5. Predicate

set of computational operations applied to a data instance which evaluate to true or false.

Note 1 to entry: In relational algebra, this is called a selection.

[SOURCE: [5]]

3.1.6. property selection

operation to create a copy of a data instance, restricted to a subset of the properties of the data instance.

Note 1 to entry: In relational algebra, this is called a projection. This term was used in the OGC Web Feature Service standard, but the Features API SWG has decided not to use the term, because in the context of geographic information the term “projection” is closely associated with map projections and causes confusion if used with a different meaning.

Note 2 to entry: This topic is one of the future parts that the Features API SWG is starting to work on. See the initial proposal.

3.1.7. query

request for data from a dataset.

Note 1 to entry: A query will at least identify the data that the query operates on, the predicate(s) used to select the result set, the properties of the data instances that should be included in the response, the order in which the data instances should be included in the response, and the maximum number of data instances in the response.

3.1.8. queryable

a token that represents a property of a resource that can be used in a filter expression.

[SOURCE: [14]]

3.2. returnable

a token that represents a property of a resource that can be included in a representation of the resource.

Note 1 to entry: The term has been introduced in the proposal for the Schemas extension of OGC API Features, so far without a definition.

Note 2 to entry: APIs implementing OGC API Features will include all returnables in a response unless the property has no value for the instance or if the property is not included in the list of requested properties (see property selection).

3.3.  sortable

a token that represents a property of a resource that can be used to sort a collection of resource instances.

NOTE  The term has been introduced in OGC API Records without a definition. Note that the content will be moved from OGC API Records to a new part of OGC API Features.

3.3.1. sorting

operation to order the data instances in a set based on the values of selected properties of each data instance.

Note 1 to entry: This capability is currently part of OGC API Records, but will be moved to a new part of OGC API Features.

3.3.2. Standardized API

an API that is intended to be deployed by multiple API providers with the same API definition.

Note 1 to entry: The only difference between the API definitions will be the URL(s) of the API deployment. All other aspects are identical (resources, content schemas, content constraints and business rules, content representations, parameters, etc.) so that any client that can use one deployment of the standardized API definition can also use all other deployments, too.

Note 2 to entry: If the API provides access to data, different deployments of the API will typically share different content.

3.3.3. Standards-based API

an API that conforms to one or more conformance classes specified in one or more standards.

Note 1 to entry: Since almost all APIs will conform to some standard, the term is usually used in the context of a specific standard or a specific family of standards. This ER considers Web APIs with a specific focus on the OGC API standards. Therefore, whenever the term is used in this ER, it is meant as an alias for an API that conforms to one or more conformance classes as defined in the OGC API standards.

3.3.4. stored query

a predefined query that is available a resource in a Web API.

Note 1 to entry: Stored queries can be used for two purposes. The first is to save users of the API the effort of creating their own queries. The second is to constrain what users may receive and how. The second purpose was the main purpose in the testbed where a developer creates stored queries for use by business users.

Note 2 to entry: The Testbed 18 requirements state that “filtering rules” must be defined “in some machine readable way.” This Engineering Report uses stored query in the OGC standards and filtering rule in the Testbed 18 requirements as synonyms. The “JSON file with filtering rules” in Figure 11 of the Testbed 18 Call for Participation is a query encoded as a JSON object that includes filter expressions using CQL2-Text or CQL2-JSON.

3.3.5. SWIM Data

any data provided through the SWIM System.

3.3.6. Web API

an API using an architectural style that is founded on the technologies of the Web [9].

Note 1 to entry: Best Practice 24: Use Web Standards as the foundation of APIs in the W3C Data on the Web Best Practices [9] provides more detail.

Note 2 to entry: A Web API is basically an API based on the HTTP standard(s).

3.4.  Abbreviated terms

AIXM

Aeronautical Information Exchange Model

API

Application Programming Interface

CQL2

OGC Common Query Language

CRS

Coordinate Reference System

ER

Engineering Report

FAA

Federal Aviation Administration

JSON

JavaScript Object Notation

NAS

National Airspace System

NOTAM

Notice to Airmen

OGC

Open Geospatial Consortium

SFDPS

SWIM Flight Data Publication Service

SWIM

System Wide Information Management

TB

Testbed

TIE

Technology Integration Experiment

WFS

Web Feature Service

4.  Introduction

4.1.  Background

4.1.1.  SWIM

The System-Wide Information Management (SWIM) initiative supports the sharing of aeronautical, air traffic, and weather information. This is accomplished by providing communications infrastructure and architectural solutions for identifying, developing, provisioning, and operating a network of highly distributed, interoperable, and reusable services.

As part of the SWIM architecture, data providers create services for consumers to access their data. Each service is designed to be stand-alone. However, the value of data increases when combined with other data. Real-world situations are often not related to data from one but instead from several SWIM feeds. The need for consumers to retrieve data from several SWIM services creates the need of interoperability between those services.

4.1.2.  OGC API Standards

For several years, the OGC members have worked on developing a family of OGC Web API standards for various geospatial resource types. These OGC API Standards are defined using OpenAPI. As the OGC API standards keep evolving, are approved by the OGC, and are implemented by the community, the aviation industry can subsequently experiment and implement them.

The following OGC API Standards and Draft Specifications were used for the development of APIs during Testbed 18.

OGC API – Features: A multi-part standard that defines the capability to create, modify, and query vector feature data on the Web and specifies requirements and recommendations for APIs to follow a standard way of accessing and sharing feature data. It currently consists of the following four parts.

  • OGC API — Features — Part 1: Core. Approved September 2019, this standard defines discovery and query operations. [12]

  • OGC API — Features — Part 2: Coordinate Reference Systems by Reference. This standard, approved October 2020, extends the core capabilities specified in Part 1: Core with the ability to use coordinate reference system (CRS) identifiers other than the defaults defined in the core. [13]

  • Draft OGC API — Features — Part 3: Filtering. Part 3 specifies an extension to the OGC API — Features — Part 1: Core standard that defines the behavior of a server that supports enhanced filtering capabilities. [14]

  • Draft OGC API — Features — Part 4: Create, Replace, Update, and Delete. Part 4 specifies an extension that defines the behavior of a server that supports operations to add, replace, modify, or delete individual resources from a collection. [15]

  • Proposal OGC API — Features — Part 5: Search. The proposal is an initial draft for Query resources that support queries on multiple collections in the same request, parameterized stored queries and join queries. [6]

A Common Query Language (CQL2) is being developed together with Part 3 to standardize a language that is recommended for filter expressions. [16]

OGC API – Processes: An approved (August 2021) OGC API Standard, specifies requirements for implementing a Web API that enables the execution of computing processes and the retrieval of metadata describing their purpose and functionality. Typically, these processes combine raster, vector, coverage, and/or point cloud data with well-defined algorithms to produce new information. [1]

Draft OGC API – Tiles: This recent OGC API Standard defines how to discover which resources offered by the Web API can be retrieved as tiles, retrieve metadata about the tile set (including the supported tile matrix sets, the limits of the tiled set inside the tile matrix set), and how to request a tile. [2]

Draft OGC API – Styles: This draft OGC API specifies building blocks for implementing OGC Standards based Web APIs that enable map servers, clients, and visual style editors to manage and fetch styles. [3]

4.1.3.  Exploration of OGC API Standards by SWIM

Over the years, the FAA and the OGC have jointly explored making SWIM data more easily accessible and more valuable. As part of these past efforts, Testbed-16 brought together previous work on the development of OGC APIs, the use of semantics to enrich data, and SWIM data processing. The objectives were to deliver the first demonstration of an OpenAPI-based API serving SWIM data, a component generating aviation Linked Data, and two client applications querying and displaying that data [15].

Two of TB-16 recommendations were to integrate OGC API requirement classes within SWIM Data Services and to demonstrate interoperability between diverse Aviation APIs [15]. In order to advance these recommendations, TB-17 focused on the development of eleven APIs based on OGC API Standards and the completion of Technology Integration Experiments (TIEs) between these APIs.

During TB-16, the development of the API serving aviation data resulted in numerous lessons learned and recommendations [15]. TB-16 saw the development of one aviation-related API based on an OGC API Standard (OGC API — Features). The APIs developed during TB-17 ([4]) addressed many of those lessons learned and implemented additional OGC API Standards (draft and approved) which have been maturing since. This process is reflected in Figure 2.

Figure 2 — History of OGC experiments to enhance SWIM

4.2.  Requirements Statement

Testbed-18 required investigating the potential of filtering using OGC API Standards in the context of the SWIM Program.

The original goals of the TB-18 Advanced Filtering of SWIM Feature Data Task were as follows.

  • Experiment with OGC API-Features filtering mechanisms.

  • Explore if best practices for advertising filtering capabilities are required beyond what is already defined in the various OGC API-Features Parts.

  • Demonstrate advanced filtering in situations where the data endpoints support only rudimentary filtering by introducing a new service type “Filtering Service.”

  • Allow filtering rules for a specific data service to be provided at runtime in a machine-readable manner.

The research questions for the Advanced Filtering of SWIM Feature Data Task were as follows.

  • How does filtering of SWIM data served by OGC API-Features endpoints work?

  • Is the metadata required by the various OGC API-Features parts sufficient to allow clients to fully “understand” the filtering capabilities of a service endpoint?

  • OGC API — Features — Part 3: Filtering and the Common Query Language (CQL) supports queryables that are not directly represented as resource properties in the content schema of the resource. Is it possible to identify best practices for their usage?

  • Clients may know the content schema of offered resources. How best to use this knowledge for advanced filtering beyond what is defined in OGC API — Features — Part 3: Filtering and the Common Query Language (CQL)?

  • How does a filtering service look that allows advanced filtering for rather simple OGC API-Features-based SWIM data endpoints?

  • How does such a service work in situations where a data publisher has restricted filtering on certain properties (for example, because the backend datastore has not been configured to allow high-performance queries on those properties)?

  • How can a client application support a customer who has knowledge of the content schema of an offered resource in the creation of filter statements? What are the key requirements for a developer GUI that supports visualization and management of these filtering tools?

  • Is it possible to easily create a new filtered dataset by creating machine readable filtering rules based on the metadata required by the OGC API-Features standards? How can these rules be provided to the Filtering Service at runtime?

4.3.  Functional Overview

As shown in Figure 3, the Advanced Filtering of SWIM Feature Data Task architecture was organized into a system of seven interconnected components. All seven components were developed simultaneously throughout the Testbed, with permanent communication and cooperation among participant organizations.

The components can be divided into the following three groups.

  • Façades for SWIM services with simple filtering mechanisms. Retrieve aviation data from multiple SWIM services and serve these data through APIs built based on OGC API Standards featuring basic filtering mechanisms. Three Façades were built.

    • The OGC API-Features Façade 1 (identified collectively as D100): Four APIs built to serve NOTAMs, Airport Layouts, and Airspaces

    • The OGC API-Features Façade 2 (identified collectively as D101): Three APIs built to serve aeronautical, flight, and weather features.

    • An extra façade, not originally included in the Task architecture, was offered in-kind by the company Skymantics, and was named OGC API-Features Façade 3: An API built to serve flight plans from the SFDPS (FAA) Service.

  • Components that serve aviation data with advanced filtering mechanisms. Two filtering services were built, each one featuring an API.

    • The Filtering Service 1 (identified as D102): Built to serve SWIM data from D100 with advanced filtering mechanisms.

    • The Filtering Service 2 (identified as D103): Built to serve SWIM data from all three façades with advanced filtering mechanisms.

  • Client components to demonstrate consumption of filtered data and configuration of filtering mechanisms. Two clients were built: One meant to serve an aviation domain expert and the other to serve a developer of aviation software applications.

    • The Business User Client (identified as D104): A client built to query filtering services and demonstrate the usage of advanced filtering mechanisms.

    • The Developer Client (identified as D105): A client built to define filter statements that can be expressed in a machine-readable way and exchanged with the filtering services.

Figure 3 — Component Diagram for the Advanced Filtering of SWIM Feature Data Task

4.3.1.  Component Interactions

The following two figures illustrate the intended interactions between the components described in the Work Items & Deliverables section of this ER. The two figures illustrate the workflows for using the filtering service for data subsetting (Figure 4) from the perspective of a business client and for configuring the filtering service at runtime (Figure 6) from the perspective of the filtering rules developer.

In the first workflow, illustrated in Figure 4, an OGC API-Features façade to SWIM Data Service data service offers insufficient filtering capabilities to its customers. The Business User Client does not want to access large data sets and then perform filtering itself. Instead, the client wants to make use of a Filtering Service that can handle the filtering of the data and provide the subset of the data that the client is interested in. If the filtering service receives a data request from the client, it connects to the data service to access the necessary data, filters out everything that is not requested by the client, and eventually delivers the result to the client.

Figure 4 — Workflow from the perspective of a business user that needs filtered data

Figure 5 — First Workflow Sequence Diagram

The second workflow, illustrated in Figure 6, demonstrates how a filtering service can be configured at run time. The assumption is that the Developer Client is aware of the API characteristics of the data service as well as the content schema of the data served by the data server. Based on both, the client supports the user with a GUI in the definition of the filtering rules. The user can then register these rules with the filtering service, which is now configured to run the data service specific filtering.

Figure 6 — Workflow from the perspective of a filtering rules developer

Figure 7 — Second Workflow Sequence Diagram

5.  Operations

Table 1 — Operations

EndpointMethodRequestResponseDescription
/searchGETn/aList of stored queriesFetch the stored queries on the server
/searchPOSTA query expressionA feature collectionExecute an ad-hoc query
/search/{queryId}GETn/aA feature collectionExecute the stored query; parameters are submitted as query parameters
/search/{queryId}POSTURL-encoded form with query parametersA feature collectionExecute this stored query
/search/{queryId}PUTA query expressionn/aCreate or update a stored query
/search/{queryId}DELETEn/an/aDelete this stored query
/search/{queryId}/definitionGETn/aA query expressionGet the definition of the stored query
/search/{queryId}/parametersGETn/aJSON Schema of an object where each parameter is a propertyGet the definition of the parameters
/search/{queryID}/parameters/{parameterID}GETn/aJSON Schema of the parameterGet the details of a query parameter

5.1.  Conformance Classes

  • Core: Support executing stored queries.

  • Parameterized Stored Queries: Support executing parameterized stored queries.

  • Manage Stored Queries: Support reading, creating, replacing, and deleting stored queries.

  • Ad-hoc Queries: Support executing ad-hoc queries.

Table 2 — Conformance Classes

EndpointMethodConformance Class
/searchGETCore
/searchPOSTAd-hoc Queries
/search/{queryId}GETCore
/search/{queryId}POSTCore
/search/{queryId}PUTManage Stored Queries
/search/{queryId}DELETEManage Stored Queries
/search/{queryId}/definitionGETManage Stored Queries
/search/{queryId}/parametersGETParameterized Stored Queries
/search/{queryID}/parameters/{parameterID}GETParameterized Stored Queries

Note that /search/{queryId}/definition is part of the ‘Manage Stored Queries’ Conformance Class. This is because in the current design GET on /search returns only the main query metadata and, if applicable, the parameter descriptions. This supports use cases where a query provider wants to keep the query expressions hidden from regular users. That is, the query expression of a stored query should only be visible to those managing the query. Consequently, GET on /search/{queryId}/definition is part of the Manage Stored Queries conformance class.

6.  Schemas

6.1.  Query Expression

{
 
"$schema": "https://json-schema.org/draft/2019-09/schema",
 
"$id": "QueryExpression.json",
 
"oneOf": [
   
{
     
"allOf": [
       
{"$ref": "#/$defs/query"},
       
{
         
"type": "object",
         
"properties": {
           
"title": {"type": "string"},
           
"description": {"type": "string"},
           
"limit": {"$ref": "#/$defs/limit"},
           
"parameters": {"$ref": "Parameters.json"}
         
}
       
}
     
]
   
},
   
{
     
"type": "object",
     
"required": ["queries"],
     
"properties": {
       
"title": {"type": "string"},
       
"description": {"type": "string"},
       
"queries": { "type": "array", "minItems": 1, "items": {"$ref": "#/$defs/query"} },
       
"filter": {"$ref": "#/$defs/filter"},
       
"filterOperator": { "type": "string", "enum": ["and", "or"], "default": "and" },
       
"properties": {"$ref": "#/$defs/properties"},
       
"limit": {"$ref": "#/$defs/limit"},
       
"parameters": {"$ref": "Parameters.json"}
     
}
   
}
 
],
 
"$defs": {
   
"query": {
     
"type": "object",
     
"required": ["collections"],
     
"properties": {
       
"collections": { "type": "array", "minItems": 1, "items": {"type": "string"} },
       
"filter": {"$ref": "#/$defs/filter"},
       
"properties": {"$ref": "#/$defs/properties"},
       
"sortby": {"$ref": "#/$defs/sortby"}
     
}
   
},
   
"filter": {"type": "object"},
   
"properties": { "type": "array", "minItems": 1, "items": {"type": "string", "minLength": 1} },
   
"sortby": {
     
"type": "array",
     
"minItems": 1,
     
"items": {"type": "string", "pattern": "[+|-]?.+"}
   
},
   
"limit": {"type": "integer", "minimum": 1, "default": 1000, "maximum": 10000}
 
}
}

A query expression can contain a single query (the properties “collections,” “filter,” “properties,” and “sortby” are members of the query expression object) or multiple queries (a “queries” member with an array of query objects is present) in a single request.

For each query

  • The value of “filter” is a CQL2 JSON filter expression.

  • The value of “properties” is an array with the names of properties to include in the response.

  • The value of “sortby” is used to sort the features in the response.

  • Multiple entries in the “collections” member represent a join between the specified collections. Just like in SQL, the properties of each collection participating in the join are combined and presented in the features in the result set. Property names in the request and in the response have to be prefixed with the collection name plus a period (“.”), e.g., “collection.property.”

    • Support for joins was a stretch goal in Testbed 18 and not discussed or tested in detail.

    • There are several open questions related to joins that should be considered by the Features API SWG.

      • How should the id of the joined feature be assigned? A combination of all features in the tuple?

      • How should the primary geometry be assigned (in the GeoJSON representation) if there are multiple geometry properties?

      • Should it also be allowed to “nest” joined properties, so instead of { …​, "col.prop1": 1, "col.prop2": "a", …​ } encode it as { …​, "col": { "prop1": 1, "prop2": "a" }, …​ }?

For multiple queries

  • If multiple queries are specified, the results are concatenated. The response is a single feature collection.

    • The feature ids in the response to a multi-collection query must be unique. Since the featureId of a feature only has to be unique per collection, they need to be combined with the collectionId. The server could determine how the id values are constructed or require a specific approach. The latter has the advantage that the source feature could be identified (if, e.g., a concatenation of the collectionId and featureId such as “apronelement.123456” is used). However, if this is a requirement, then maybe a “self” link in each feature would be cleaner? In addition, if an API already uses feature identifiers that are unique across all collections (e.g., a UUID), then a mandatory collection prefix would be unnecessary.

    • Another aspect is cases where a feature is included in the result set of multiple queries. The feature should be included only once in the result. However, if the ids are constructed in a way that there is no conflict, if the same feature is included multiple times (e.g., just an auto-incrementing index value as the id), maybe it could also be tolerated if the same feature is included more than once?

    • Another approach to both issues could be to return an array of feature collections in the response, one for each query. Similar to what was done in WFS 2.0. This seems cleaner and clearer as it avoids the “hacks” and issues discussed in the previous bullet items. Since /search is a different resource than /items this would be conceptually clean. This leaves the question of usability: a single feature collection is easier if the result is just fed into some GeoJSON tooling. However, in a JavaScript based client it seems there is no overhead to process such a response. It could also be beneficial for use with JSON-FG where each feature collection would more likely be homogeneous and could include metadata that simplify parsing the response.

  • The direct members “filter” and “properties” represent “global” constraints that must be combined with the corresponding member in each query. The global and local property selection list are concatenated and then the global and local filters are combined using the logical operator specified by the “filterOperator” member.

    • The global member “filter” should only reference queryables that are common to all collections being queried. If a queryable is specified that is not defined or does not exist for a particular collection then the value of the property is null.

    • The global member “properties” should only reference presentables that are common to all collections being queried. If a presentable is specified that is not defined or does not exist for a particular collection then the property is omitted from the response.

General rules

  • A “title” and “description” for the query expression can be added. Providing both is strongly recommended to explain the query to users.

  • The “limit” member applies to the entire result set.

  • Note that “sortby” will only apply per query. A global “sortby” would require that the results of all queries are compiled first and then the combined result set is sorted. This would not support “streaming” the response.

  • In case of a parameterized stored query, the query expression may contain JSON objects with a member “$parameter.” The value of “$parameter” is an object with a member where the key is the parameter name and the value is a JSON schema describing the parameter. When executing the stored query, all objects with a “$parameter” member are replaced with the value of the parameter for this query execution. Comma-separated parameter values are converted to an array if the parameter is of type “array”.

  • Parameters may also be provided in a top-level member “parameters” and referenced using “$ref”.

6.2.  Stored Queries

{
 
"$schema": "https://json-schema.org/draft/2019-09/schema",
 
"$id": "StoredQueries.json"
 
"type": "object",
 
"properties": {
   
"queries"    : { "type": "array" , "items": {"$ref": "#/$defs/StoredQueryDescription"} },
   
"links"      : { "type": "array" , "items": {"$ref": "#/$defs/Link"}                   },
   
"title"      : { "type": "string"                                                      },
   
"description": { "type": "string"                                                      }
 
},
 
"$defs": {
   
"StoredQueryDescription": {
     
"required": ["id"],
     
"type": "object",
     
"properties": {
       
"id": {"type": "string"},
       
"title": {"type": "string"},
       
"description": {"type": "string"},
       
"parameters": { "$ref": "Parameters.json" },
       
"links": { "type": "array", "items": {"$ref": "#/$defs/Link"} }
     
}
   
},
   
"Link": {
     
"type": "object",
     
"required": ["href", "rel"],
     
"properties": {
       
"href": {"type": "string"},
       
"rel": {"type": "string"},
       
"title": {"type": "string"},
       
"type": {"type": "string"},
       
"hreflang": {"type": "string"},
       
"length": {"type": "integer"},
       
"templated": {"type": "boolean", "default": false}
     
}
   
}
 
}
}

6.3.  Parameters

The parameters are described as a JSON object where each parameter is a property with its JSON Schema as its value.

Providing sufficient information that allows clients to generate meaningful queries is essential. Recommendation 1 in OGC API — Features — Part 3 has recommendations for schemas that are straightforward to parse by clients but that are expressive enough to allow clients to generate forms to provide parameter values.

In addition, if a parameter declares a default value, the API will use that default value if no value is provided in the request to execute a query.

{
 
"$schema": "https://json-schema.org/draft/2019-09/schema",
 
"$id": "Parameters.json"
 
"type": "object",
 
"description": "Each parameter is described by a property where each value is a JSON Schema object.",
 
"additionalProperties": {
   
"$ref": "https://json-schema.org/draft/2019-09/schema"
 
}
}

7.  Processes — Part 3 approach

7.1.  Pre-defining queries based on Processes — Part 3 extension

The OGC API — Features Search extension (described in detail in the other section of this document) shares a lot in common with the idea of allowing filter, properties (for selection and derived fields/properties), and sortBy to qualify inputs in OGC API — Processes — Part 3: Workflows and Chaining (input and output modifiers requirements classes).

A well-known pass-through process (with support for collection input including filtering) could support an execution request with a syntax equivalent to the Search extension endpoint, similar to how the OGC API — Routes /routes endpoint shares a POST payload syntax with an eventual definition of a well-known routing process. Pre-defined queries could also be parameterized by deploying them as processes, as suggested in the Deployable workflows requirements class of Processes — Part 3: Workflows and Chaining.

The modifiers introduced include the same filter, properties, and sortBy parameters to qualify inputs originating from a data collection or process, whether they are local or remote, as well as outputs resulting from a process. In addition to the ability to select specific fields/properties, the properties parameter can also be used to derive new fields, for example using CQL2 arithmetic expressions.

Processes — Part 3 also defines a Collection output requirements class where the output of the workflow execution is either a dataset landing page (which can contain multiple collections), or a single collection.

For example, the following parameterized query, addressing similar use case as the one described further in section 8.3.5 — Support for the Search resources‘s Example 2, taking two parameters, a string enumeration named composition and a string array named airports, could be expressed in a Part 3-extended OGC API — Processes execution request (using Collection input and Output modifiers) as shown below.

Example 1 — Example parameterizable query as a Part 3-extended execution request

{
 
"process" : "PassThrough",
 
"inputs" : {
 
"data" : [ {
   
"collection" : "apronelement",
   
"filter": {
     
"op": "and",
     
"args": [
       
{
         
"op": "=",
         
"args": [
           
{"property": "composition"},
           
{
             
"$input": {
               
"composition": { "type": "string", "enum": ["CONC", "..."] }
             
}
           
}
         
]
       
},
       
{
         
"op": "in",
         
"args": [
           
{"property": "airport"},
           
{
             
"$input": {
               
"airports": {
                 
"type": "array",
                 
"items": { "type": "string", "enum": ["JFK", "EWR", "LGA", "..."] }
               
}
             
}
           
}
         
]
       
}
     
]
   
},
   
"properties": ["geometry", "airport", "type"],
   
"sortBy": ["airport"]
 
} ]
 
}
}

In this example, data is an input defined in the PassThrough well-known process with multiplicity 1..* which is returned as the process output.

The collection property is defined in the Collection input requirements class of OGC API — Processes — Part 3, whereas the filter, properties, and sortby elements specifying a cql2-json filtering expression, selected properties and an ascending sort order by airport, are defined in Part 3’s Input modifiers requirements class.

These field modifiers can also be used in the context of the Output modifiers requirements class together with the Collection output requirement class or with regular process execution outputs. In the context of a feature collection output, these query parameters building blocks also correspond to the functionality provided by OGC API — Features — Part 3: Filtering, as well as a planned extensions for Coverages and Discrete Global Grid Systems.

This execution request could be deployed as a new process, e.g., ApronFiltering, using the Processes — Part 2: Deploy, Replace, Undeploy extension’s POST operation to /processes together with the Deployable workflows requirements class of Part 3. The resulting process would get listed at /processes with a process description including the input parameters (composition and airports) and could itself be executed by POSTing to /processes/ApronFiltering/execution.

Example 2 — Example execution request of parameterized query deployed as a process

{
 
"inputs" : {
   
"composition" : "CONC",
   
"airports": [ "JFK", "LGA" ]
 
}
}

Posting this execution request to the execution endpoint without a Prefer: header would result in a synchronous execution that returns the features. With support for the Collection output requirements class, specifying as parameter response=collection would instead return a collection description, as for a GET request to /collections/{collectionId} in Common — Part 2 and Features — Part 1. With response=landingPage, a landing page would be returned for the filtered dataset, allowing to retrieve multiple collections.

Such virtual collections could also be published to a dataset API as a persistent collection, e.g., as /collections/concApronsJFKLGA, using a non-parameterizable execution request as the payload of a POST operation to /collections to create the dynamic collection, with a Processes execution request content media type to be registered, e.g., application/ogcexecreq+json as suggested in Part 3 — Section 14. Media Types.

During this Testbed-18 advanced filtering task Ecere successfully demonstrated the use of Part 3 extensions to pre-define filtering queries to the cascading service, including the deployment of a sample PassThrough process with support for the filter and properties modifiers, as well as for the Collection Input and Collection output requirements class, with the following limitations:

  • the filters were expressed using the cql2-text encoding rather than cql2-json;

  • the input and output were limited to a single collection;

  • the sortBy modifier (and sorted feature collections in general) remained to be implemented; and

  • support for parameterized queries and deployable workflows was not yet implemented.

A sample pre-defined query is available from the endpoint:

https://maps.gnosis.earth/ogcapi/processes/PassThrough/execution?response=collection

including the following default pre-defined filter execution request:

Example 3 — Working execution request of filtering query from D100 cascaded collection

{
  
"process" : "https://maps.gnosis.earth/ogcapi/processes/PassThrough",
  
"inputs" : {
     
"data" : [
        
{
           
"collection" : "https://maps.gnosis.earth/ogcapi/collections/swim:d100_airports:apronelement",
           
"filter" : "composition = 'CONC' and airport in ('JFK', 'EWR', 'LGA')",
           
"properties" : [ "geometry", "airport", "type" ]
        
}
     
]
  
}
}