I. Abstract
This OGC Testbed-18 (TB-18) Features Filtering Summary Engineering Report (ER) summarizes the implementations, findings, and recommendations that emerged from the efforts to better understand the current OGC API-Features filtering capabilities and limitations and how filtering can be decoupled from data services.
This ER describes:
two façades built to interface SWIM services and serve aviation data through APIs (built with OGC API Standards) including basic filtering capabilities;
the two filtering services built to consume SWIM data and serve it through OGC based APIs featuring advanced filtering mechanism;
the client application built to interface with the filtering services; and
the developer client built to define filter statements that can be expressed in a machine-readable way and exchanged with the filtering service.
II. Executive Summary
Previous OGC work has addressed the challenges of increasing interoperability between aviation data services. Recently, the OGC community has developed a new family of standardized OpenAPI-based Web API Standards for various geospatial resource types. These OGC APIs have the potential to enhance the way in which consumers can access geospatial data from various sources. Testbed-16 brought together previous work on the development of OGC API Standards, the use of semantics to enrich data, SWIM data processing, and demonstrated an OpenAPI-based API serving SWIM data. Testbed-17 used the lessons learned and recommendations from Testbed-16 and focused on further testing the value of standards-based APIs within the SWIM program.
OGC API-Features endpoints define their filtering capabilities. Filtering is standardized across different parts of OGC API-Features (see section Previous Work), with two parts still in draft status. Advanced filtering capabilities require sophisticated server software. Not all data providers will be able to operate such a powerful service endpoint. FAA SWIM Data Services currently produce data from the National Airspace System (NAS) that is provided to consumers using various protocols and service offerings in both synchronous and asynchronous messaging formats. Testbed-18 (TB-18) explored filtering mechanisms for feature data served by OGC API-Features instances. The experiments included filtering of native and fused SWIM data and experimented with filtering services.
The research questions for the Advanced Filtering of SWIM Feature Data Task were as listed below.
How does filtering of SWIM data served by OGC API-Features endpoints work?
Is the metadata required by the various OGC API-Features parts sufficient to allow clients to fully understand the filtering capabilities of a service endpoint?
OGC API — Features — Part 3: Filtering and the Common Query Language (CQL) supports queryables that are not directly represented as resource properties in the content schema of the resource. Is it possible to identify best practices for their usage?
Clients may know the content schema of offered resources. How should this knowledge be used for advanced filtering beyond what is defined in particular in OGC API — Features — Part 3: Filtering and the Common Query Language (CQL)?
How does a filtering service that allows advanced filtering for rather simple OGC API-Features-based SWIM data endpoints look like?
How does such a service work in situations where a data publisher has restricted filtering on certain properties (for example, because the backend datastore has not been configured to allow high-performance queries on those properties)?
How can a client application support a customer that has knowledge of the content schema of an offered resource in the creation of filter statements? What are the key requirements for a developer GUI that allows visualization and management of these filtering tools?
Is it possible to easily create a new filtered dataset by creating machine readable filtering rules based on the metadata required by the OGC API-Features standards? How can these rules be provided to the Filtering Service at runtime?
To answer these questions, this Testbed-18 Task was organized into the development and testing of a system of six interconnected components, as seen on Figure 1 and listed below.
Façades for SWIM services with simple filtering mechanisms, retrieving aviation data from multiple SWIM services, serving these data through APIs built based on OGC API standards, and featuring basic filtering mechanisms. Three Façades were built:
the OGC API-Features Façade 1 (identified collectively as D100): Four APIs built to serve NOTAMs, Airport Layouts, and Airspaces;
the OGC API-Features Façade 2 (identified collectively as D101): Three APIs built to serve aeronautical, flight, and weather features; and
an extra façade, not originally included in the Task architecture, offered in-kind by the company Skymantics, and was named OGC API-Features Façade 3: An APIs built to serve flight plans from the SFDPS (FAA) Service.
Components that serve aviation data with advanced filtering mechanisms. Two filtering services were built, each one featuring an API:
the Filtering Service 1 (identified as D102): Built to serve SWIM data from D100 with advanced filtering mechanisms; and
the Filtering Service 2 (identified as D103): Built to serve SWIM data from all three façades with advanced filtering mechanisms.
Client components to demonstrate consumption of filtered data, and configuration of filtering mechanisms. Two clients were built: One meant to serve an aviation domain expert, and the other to serve a developer of aviation software applications:
the Business User Client (identified as D104): A client built to query filtering services and demonstrate the usage of advanced filtering mechanisms; and
the Developer Client (identified as D105): A client built to define filter statements that can be expressed in a machine-readable way and exchanged with the filtering services.
Figure 1 — Component Diagram for the Advanced Filtering of SWIM Feature Data Task
All components were successfully developed and tested. The lessons learned throughout the Testbed are documented in this ER and help respond to the questions posed above. The following is a set of recommendations for future work.
Technical Design and CQL Standard: The implementations in this Testbed provided feedback to the technical design and helped to improve the specification in Testbed-18 Filtering Service and Rule Set Engineering Report (D002) as well as in the Common Query Language (CQL2) candidate standard.
Filter close to the data: In this task, the deliverables architecture called for one component performing the filtering, while another acts as an OGC API façade. The Testbed initiative highlighted that this architecture is not ideal, suggesting the exploration of filtering performed as close to the source of data as possible.
Explore the potential of spatial joins: Future work could explore, in the context of queries spanning multiple collections, the capability of performing queries that not only filter and combine the results from multiple collections, but also perform join operations on them, including spatiotemporal joins.
First-Filtering: Future work could explore how the filtering service might be able to perform a first-filtering pass that requires retrieving less data from the original data source.
Interactive Query Building Interface: Future initiatives could explore an Interactive query builder web interface providing graphical and pre-selectable queryables, sortables, and conformances that are fetched from the SWIM Filtering service. This has the potential to speed query building and validation.
Smart Query Building Interface: Another shortcoming is the lack of real-time hints on the impact of changes to the query structure and on the results of the query. Having graphical query results that are tied back to the query expression helps the user understand the impact of each queryable or conformance used in the query as well as the result of the query. The results of the query would be graphically displayed in real-time to the Developer User while the query is being built, helping the Developer User to more quickly adjust the query to achieve the desired results.
Data Correlation from multiple SWIM Services: In TB-18 the queries built using CQL2 language are limited to each SWIM data service, and these queries are segregated from each other. Future work should explore filtering services supporting data fusion based on parameters in the data that are of similar type that support the same conformances.
III. Keywords
The following are keywords to be used by search engines and document catalogues.
testbed, web service, api, standard, filter, SWIM, aviation
IV. Preface
It is possible that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.
Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed upon by any implementation of the standard set forth in this document and to provide supporting documentation.
V. Security considerations
No security considerations have been made for this document.
VI. Submitters
All questions regarding this document should be directed to the editor or the contributors:
| Name | Organization | Role |
| Sergio Taleisnik | Skymantics, LLC | Editor |
| Clemens Portele | interactive instruments GmbH | Contributor |
| Eugene Yu | George Mason University | Contributor |
| Jérôme Jacovella-St-Louis | Ecere Corporation | Contributor |
| Patrick Dion | Ecere Corporation | Contributor |
| Mohammad Moallemi | Concepts Beyond LLC | Contributor |
Testbed-18: Features Filtering Summary Engineering Report
1. Scope
This OGC Testbed 18 Engineering Report (ER) summarizes the implementations, findings, and recommendations that emerged from Testbed-18 efforts regarding the current filtering capabilities and limitations with the OGC API — Features Standard and how filtering can be decoupled from data services. The ER describes two façades built to interface SWIM services and serve aviation data through APIs built using OGC API Standards.
2. Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.
Open API Initiative: OpenAPI Specification 3.0.2, 2018 https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md
van den Brink, L., Portele, C., Vretanos, P.: OGC 10-100r3, Geography Markup Language (GML) Simple Features Profile, 2012 http://portal.opengeospatial.org/files/?artifact_id=42729
W3C: HTML5, W3C Recommendation, 2019 http://www.w3.org/TR/html5/
Schema.org: http://schema.org/docs/schemas.html
R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee: IETF RFC 2616, Hypertext Transfer Protocol — HTTP/1.1. RFC Publisher (1999). https://www.rfc-editor.org/info/rfc2616.
E. Rescorla: IETF RFC 2818, HTTP Over TLS. RFC Publisher (2000). https://www.rfc-editor.org/info/rfc2818.
G. Klyne, C. Newman: IETF RFC 3339, Date and Time on the Internet: Timestamps. RFC Publisher (2002). https://www.rfc-editor.org/info/rfc3339.
M. Nottingham: IETF RFC 8288, Web Linking. RFC Publisher (2017). https://www.rfc-editor.org/info/rfc8288.
H. Butler, M. Daly, A. Doyle, S. Gillies, S. Hagen, T. Schaub: IETF RFC 7946, The GeoJSON Format. RFC Publisher (2016). https://www.rfc-editor.org/info/rfc7946.
3. Terms, definitions and abbreviated terms
This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.
This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.
For the purposes of this document, the following additional terms and definitions apply.
3.1. Terms and definitions
3.1.1. Application Programming Interface (API)
an interface that is defined in terms of a set of functions and procedures, and enables a program to gain access to facilities within an application [7].
3.1.2. Façade Service
a component that fetches data from a specific data source and makes it available through its own interface [9]. The main reason for building this type of service is the difficulty or inability to modify the original data source with the intent of modifying either:
the underlying structure of the API; or
the format in which the data is made available.
3.1.3. Standardized API
an API that is intended to be deployed by multiple API providers with the same API definition.
Note 1 to entry: The only difference between the API definitions will be the URL(s) of the API deployment. All other aspects are identical (resources, content schemas, content constraints, business rules, content representations, parameters, etc.) so that any client that can use one deployment of the standardized API definition can also use all other deployments, too.
Note 2 to entry: If the API provides access to data, different deployments of the API will typically share different content.
3.1.4. Standards-based API
an API that conforms to one or more conformance classes specified in one or more standards.
Note 1 to entry: Since almost all APIs will conform to some standard, the term is usually used in the context of a specific standard or a specific family of standards. This ER considers Web APIs with a specific focus on the OGC API standards. Therefore, whenever the term is used in this ER, it is meant as an alias for an API that conforms to one or more conformance classes as defined in the OGC API standards.
3.1.5. SWIM Data
any data provided through the SWIM System.
3.1.6. Web API
an API using an architectural style that is founded on the technologies of the Web [8].
Note 1 to entry: Best Practice 24: Use Web Standards as the foundation of APIs in the W3C Data on the Web Best Practices [8] provides more detail.
Note 2 to entry: A Web API is basically an API based on the HTTP standard(s).
3.2. Abbreviated terms
AIXM
Aeronautical Information Exchange Model
API
Application Programming Interface
CRS
Coordinate Reference System
ER
Engineering Report
FAA
Federal Aviation Administration
FIXM
Flight Information Exchange Model
NAS
National Airspace System
NOTAM
Notice to Airmen
OGC
Open Geospatial Consortium
SCDS
SWIM Cloud Distribution Service
SFDPS
SWIM Flight Data Publication Service
SWIM
System Wide Information Management
TB
Testbed
TFMS
Traffic Flow Management System
TIE
Technology Integration Experiment
WFS
Web Feature Service
WXXM
Weather Information Exchange Model
4. Introduction
4.1. Background
4.1.1. SWIM
The System-Wide Information Management (SWIM) initiative supports the sharing of aeronautical, air traffic, and weather information. This is accomplished by providing communications infrastructure and architectural solutions for identifying, developing, provisioning, and operating a network of highly distributed, interoperable, and reusable services.
As part of the SWIM architecture, data providers create services for consumers for data access. Each service is designed to be stand-alone. However, the value of data increases when combined with other data. Real-world situations are often not related to data from one but instead from several SWIM feeds. Since consumers can retrieve data from several SWIM services there is the need for interoperability between the services.
4.1.2. OGC API Standards
For several years, OGC members have worked on developing a family of OGC Web API standards for various geospatial resource types. These OGC API Standards are defined using OpenAPI. As the OGC API standards continue to evolve, are approved by the OGC, and are implemented by the community, the aviation industry can subsequently experiment with and implement them.
The following OGC API Standards and Draft Specifications were used for the development of APIs during Testbed 18.
OGC API – Features: A multi-part standard that defines the capability to create, modify, and query vector feature data on the Web and specifies requirements and recommendations for APIs to follow a standard way of accessing and sharing feature data. It currently consists of four parts and a fifth proposed part.
OGC API — Features — Part 1: Core. Approved September 2019, this standard defines discovery and query operations. [11]
OGC API — Features — Part 2: Coordinate Reference Systems by Reference. This standard, approved October 2020, extends the core capabilities specified in Part 1: Core with the ability to use coordinate reference system (CRS) identifiers other than the defaults defined in the core. [12]
Draft OGC API — Features — Part 3: Filtering. Part 3 specifies an extension to the OGC API — Features — Part 1: Core standard that defines the behavior of a server that supports enhanced filtering capabilities. [13]
Draft OGC API — Features — Part 4: Create, Replace, Update, and Delete. Part 4 specifies an extension that defines the behavior of a server that supports operations to add, replace, modify, or delete individual resources from a collection. [14]
Proposal OGC API — Features — Part 5: Search. The proposal is an initial draft for query resources that support queries on multiple collections in the same request, parameterized stored queries, and join queries. [5]
A Common Query Language (CQL2) is being developed together with Part 3 to standardize a language that is recommended for filter expressions. [15]
OGC API – Processes: An approved (August 2021) OGC API Standard that specifies requirements for implementing a Web API that enables the execution of computing processes and the retrieval of metadata describing their purpose and functionality. Typically, these processes combine raster, vector, coverage, and/or point cloud data with well-defined algorithms to produce new information. [1]
Draft OGC API – Tiles: This recent OGC API Standard defines how to discover which resources offered by the Web API can be retrieved as tiles, retrieve metadata about the tile set (including the supported tile matrix sets, the limits of the tiled set inside the tile matrix set), and how to request a tile. [2]
Draft OGC API – Styles: This draft OGC API specifies building blocks for implementing OGC Standards based Web APIs that enables map servers, clients, and visual style editors to manage and fetch styles. [3]
4.1.3. Exploration of OGC API Standards by SWIM
Over the years, the FAA and OGC have jointly explored making SWIM data more easily accessible and more valuable. As part of these past efforts, Testbed-16 brought together previous work on the development of OGC APIs, the use of semantics to enrich data, and SWIM data processing. The objectives were to deliver the first demonstration of an OpenAPI-based API serving SWIM data, a component generating aviation Linked Data, and two client applications querying and displaying that data [14].
Two of TB-16 recommendations were to integrate OGC API requirement classes within SWIM Data Services and to demonstrate interoperability between diverse Aviation APIs [14]. In order to advance these recommendations, TB-17 focused on the development of eleven APIs based on OGC API Standards, and the completion of Technology Integration Experiments (TIEs) between these APIs.
During TB-16, the development of the API serving aviation data resulted in numerous lessons learned and recommendations [14]. TB-16 saw the development of one aviation-related API based on an OGC API Standard (OGC API — Features). The APIs developed during TB-17 ([4]) addressed many of those lessons learned and implemented additional OGC API Standards (draft and approved) which have been maturing since. This process is reflected in Figure 2.
Figure 2 — History of OGC experiments to enhance SWIM
4.2. Requirements Statement
Testbed-18 required investigating the potential of filtering using OGC API Standards in the context of the SWIM Program.
The original goals of the TB-18 Advanced Filtering of SWIM Feature Data Task were as follows
Experiment with OGC API-Features filtering mechanisms.
Explore if best practices for advertising filtering capabilities are required beyond what is already defined in the various OGC API-Features Parts.
Demonstrate advanced filtering in situations where the data endpoints support only rudimentary filtering by introducing a new service type “Filtering Service.”
Allow filtering rules for a specific data service to be provided at runtime in a machine-readable manner.
The research questions for the Advanced Filtering of SWIM Feature Data Task were as follows.
How does filtering of SWIM data served by OGC API-Features endpoints work?
Is the metadata required by the various OGC API-Features parts sufficient to allow clients to fully “understand” the filtering capabilities of a service endpoint?
OGC API — Features — Part 3: Filtering and the Common Query Language (CQL) supports queryables that are not directly represented as resource properties in the content schema of the resource. Is it possible to identify best practices for their usage?
Clients may know the content schema of offered resources. How to use this knowledge for advanced filtering beyond what is defined in OGC API — Features — Part 3: Filtering and the Common Query Language (CQL)?
How does a filtering service look like that allows advanced filtering for rather simple OGC API-Features-based SWIM data endpoints?
How does such a service work in situations where a data publisher has restricted filtering on certain properties (for example, because the backend datastore has not been configured to allow high-performance queries on those properties)?
How can a client application support a customer who has knowledge of the content schema of an offered resource in the creation of filter statements? What are the key requirements for a developer GUI that supports visualization and management of these filtering tools?
Is easily creating a new filtered dataset possible by creating machine readable filtering rules based on the metadata required by the OGC API-Features standards? How can these rules be provided to the Filtering Service at runtime?
4.3. Functional Overview
As shown in Figure 3, the Advanced Filtering of SWIM Feature Data Task architecture was organized into a system of seven interconnected components. All seven components were developed simultaneously throughout the Testbed, with permanent communication and cooperation among participant organizations.
The components can be divided into three groups.
Façades for SWIM services with simple filtering mechanisms. Retrieve aviation data from multiple SWIM services and serve these data through APIs built based on OGC API Standards featuring basic filtering mechanisms. Three Façades were built.
The OGC API-Features Façade 1 (identified collectively as D100): Four APIs built to serve NOTAMs, Airport Layouts, and Airspaces
The OGC API-Features Façade 2 (identified collectively as D101): Three APIs built to serve aeronautical, flight, and weather features.
An extra façade, not originally included in the Task architecture, was offered in-kind by the company Skymantics, and was named OGC API-Features Façade 3: An API built to serve flight plans from the SFDPS (FAA) Service.
Components that serve aviation data with advanced filtering mechanisms. Two filtering services were built, each one featuring an API.
The Filtering Service 1 (identified as D102): Built to serve SWIM data from D100 with advanced filtering mechanisms.
The Filtering Service 2 (identified as D103): Built to serve SWIM data from all three façades with advanced filtering mechanisms.
Client components to demonstrate consumption of filtered data, and configuration of filtering mechanisms. Two clients were built: One meant to serve an aviation domain expert and the other to serve a developer of aviation software applications.
The Business User Client (identified as D104): A client built to query filtering services and demonstrate the usage of advanced filtering mechanisms.
The Developer Client (identified as D105): A client built to define filter statements that can be expressed in a machine-readable way and can be exchanged with the filtering services.
Figure 3 — Component Diagram for the Advanced Filtering of SWIM Feature Data Task
4.3.1. Component Interactions
The following two figures illustrate the intended interactions between the components described in the Work Items & Deliverables section of this ER. The two figures illustrate the workflows for using the filtering service for data subsetting (Figure 4) from the perspective of a business client and for configuring the filtering service at runtime (Figure 6) from the perspective of the filtering rules developer.
In the first workflow, illustrated in Figure 4, an OGC API-Features façade to SWIM Data Service data service offers insufficient filtering capabilities to its customers. The Business User Client does not want to access large data sets and then perform filtering itself. Instead, the client wants to make use of a Filtering Service that can handle the filtering of the data and provide the subset of the data that the client is interested in. If the filtering service receives a data request from the client, it connects to the data service to access the necessary data, filters out everything that is not requested by the client, and eventually delivers the result to the client.
Figure 4 — Workflow from the perspective of a business user that needs filtered data
Figure 5 — First Workflow Sequence Diagram
The second workflow, illustrated in Figure 6, demonstrates how a filtering service can be configured at run time. The assumption is that the Developer Client is aware of the API characteristics of the data service as well as the content schema of the data served by the data server. Based on both, the client supports the user with a GUI in the definition of the filtering rules. The user can then register these rules with the filtering service, which is now configured to run the data service specific filtering.
Figure 6 — Workflow from the perspective of a filtering rules developer
Figure 7 — Second Workflow Sequence Diagram
5. OGC API-Features Façade 1 (D100)
The OGC API-Features Façade 1, identified as deliverable 100 or D100, is a component built to demonstrate a service consuming data from a SWIM data service and serving that data through an API built based on OGC API standards offering a limited set of filtering capabilities. The component was demonstrated by interactive instruments GmbH.
The Façade Service was originally set up in Testbed 17 (see the description of D104 in chapter 4 of [4]) and updated for Testbed 18.
5.1. Internal Architecture
5.1.1. Component Overview
This Façade Service features the following.
Two data retrieval subcomponents: one for the Federal Notice to Airmen System (AIM-FNS) and another one for airspace and airport static data sources.
A database to store the extracted data.
ldproxy which delivers the APIs that serve the extracted data. Figure 8 describes the Façade Service subcomponents, the data sources (note that for AIM-FNS the retriever also accesses static sources on some occasions), and the Testbed-18 components consuming from the Façade APIs.
Figure 8 — D100 Component Overview
5.1.2. ldproxy (Web API)
The main component that delivers the Aeronautical Data APIs is ldproxy. ldproxy is a software product written in the Java programming language that implements the OGC API family of Standards. ldproxy enables sharing geospatial data using Web APIs based on OGC API Standards.
The component provides the following four Data APIs that have been deployed.
Responses with feature data could be requested in the following representations/formats.
GeoJSON (media type application/geo+json, query parameter f=json)
JSON-FG (media type application/vnd.ogc.fg+json, query parameter f=jsonfg)
HTML (media type text/html, query parameter f=html)
FlatGeobuf (media type application/flatgeobuf, query parameter f=fgb)
In addition, some of the APIs also provided access to vector tiles and styles for those tiles. These can be useful for exploring the data but are not directly related to the research questions.
For more information about the APIs, see section 4.2 of the Testbed 17 Aviation API Engineering Report.
5.1.3. PostgreSQL/PostGIS (Database)
The data available for API access are stored in a PostgreSQL/PostGIS database. That is, the data are stored in tables with the geometries stored as Simple Features geometries in Well-Known-Text format.
The following are the aeronautical datasets.
All Controlled Airspaces of the Classes B, C, D, and E from the National Airspace System Resource (NASR) Subscription
Airport data for the major airports in the United States provided by Hexagon (AIXM 5.1)
Notices to Airmen (NOTAMs) received from a Federal Notice to Airmen System (AIM-FNS) subscription in FAA’s SWIM Cloud Distribution Service (SCDS), a cloud-based infrastructure dedicated to providing near real-time FAA SWIM data to the public via Solace JMS messaging
5.1.3.1. Communication Between ldproxy and PostgreSQL
ldproxy converts API requests to SQL queries and processes the results to convert them to API responses in the GeoJSON, Features and Geometries JSON (JSON-FG), HTML, or Mapbox Vector Tile (MVT) formats.
Figure 9 — Information Flow for Data Requests
While the first two datasets are static and do not change, the NOTAMs are dynamic. New NOTAMs are added to the database as they are received from the SCDS subscription. Since a new NOTAM may change information that is cached in ldproxy for performance reasons (in particular, the spatial and temporal extent of the NOTAM dataset, but also vector tiles) a database trigger is used to notify ldproxy about a new NOTAM. For each new NOTAM, the spatial and temporal extents are evaluated and, if needed, the extents of the NOTAM feature collection are updated. Additionally, vector tiles that include the spatial extent are invalidated in the tile cache.
Figure 10 — Communicating Data Changes to ldproxy
The communication between ldproxy and PostgreSQL uses the standard PostgreSQL protocol, which is TCP/IP-based.
5.1.4. Data Retrieval
See sections 4.1.4 and 4.1.5 of the Tested-17 Aviation API Engineering Report on details how the SWIM datasets are loaded into the database.
5.2. Differences to the component from Testbed 17
The component was updated to the latest version of ldproxy and to the new version 1.0.1 of OGC API Features Part 1: Core.
The APIs mainly supported simple access to the data, based on OGC API Features Part 1: Core. That is, paging was supported as well as filtering using bbox, datetime, and selected feature attributes.
Some sample requests for data requests with simple filter capabilities were included in the description of the NOTAM API, by clicking on “Sample requests to filter NOTAM events” on the HTML landing page for more information. This gave clients an idea of how to construct data requests with simple but already useful filter capabilities.
Example — Example simple filtering requests in the NOTAM API
For spatial filtering, the “bbox” query parameter can be used. https://t18.ldproxy.net/d100_fns/collections/notam/items?bbox=-77.6,38.4,-76.2,39.6 selects events in the Washington area.
For temporal filtering, the “datetime” query parameter can be used. https://t18.ldproxy.net/d100_fns/collections/notam/items?datetime=now selects events that are currently valid, https://t18.ldproxy.net/d100_fns/collections/notam/items?datetime=2021-06-27T01:00:00Z selects events that are effective at 1AM UTC on 2021-06-27, and https://t18.ldproxy.net/d100_fns/collections/notam/items?datetime=2021-07-01T00:00:00Z/2021-07-31T23:59:59Z selects NOTAMs that are effective during July 2021.
For filtering on attributes that are queryables, a query parameter with the attribute name can be used.
https://t18.ldproxy.net/d100_fns/collections/notam/items?location=IAD selects events related to Dulles airport.
https://t18.ldproxy.net/d100_fns/collections/notam/items?icao_location=KIAD also selects events related to Dulles airport.
https://t18.ldproxy.net/d100_fns/collections/notam/items?notam_keyword=RWY selects events related to runways.
https://t18.ldproxy.net/d100_fns/collections/notam/items?affected_fir=ZDC selects events affecting the Washington Air Route Traffic Control Center (ZDC).
https://t18.ldproxy.net/d100_fns/collections/notam/items?scenario=82 selects the events related to scenario 82 (no documentation about the meaning of the scenarios could be identified).
Any of these query parameters can be combined. Events are selected that meet all predicates. https://t18.ldproxy.net/d100_fns/collections/notam/items?datetime=now=RWY=ZDC selects all NOTAMs that are currently in effect, related to runways and affecting the Washington Air Route Traffic Control Center (ZDC).
The HTML representation of the Features resource also included a simple HTML form to submit requests with these simple filters.
Figure 11 — D100 Simple Filtering in the Web Browser
In Testbed 17 the APIs supported more OGC API building blocks, in particular they supported filtering using OGC API — Features — Part 3: Filtering and the Text encoding of Common Query Language (CQL2). Since the Filtering Services D102 and D103 were used for Filtering in Testbed 18, these capabilities were removed from the Façade Service in the testbed. In addition, other capabilities were removed, like the ability to request the data in various coordinate reference systems.
5.3. Challenges and Lessons Learned
Since the component was in large part reused for Testbed 18, there were no new challenges regarding the pre-existing APIs. See section 4.3 of the Tested-17 Aviation API Engineering Report for several challenges and lessons learned during Testbed 17.
Furthermore, there was a goal to add additional weather datasets to the component in Testbed 18. However, getting access to weather data with a coverage of the continental United States within the time frame of Testbed 18 was not possible. Therefore, no additional SWIM data was added during Testbed 18 to the component.
6. OGC API-Features Façade 2 (D101)
OGC API-Features Façade 2, identified as deliverable 101 or D101, is a component built to demonstrate a service consuming data from a SWIM data service and serving that data through an API built based on OGC API standards offering a limited set of filtering capabilities. George Mason University demonstrated the D101 component.
6.1. Status Quo
For Testbed 16 (OGC document 20-020) the OGC API-Feature façade for the SWIM data messaging services was initially implemented as a RESTful relay Web service using Java, Spring framework, and Java Messaging Service (JMS) clients. A Testbed 18 objective was to test integrated filtering capabilities of OGC API-Features, especially with the OGC API-Features draft Part 3 filtering capability of OGC document 19-079r1. The following summarizes the main gaps or the advances to be made to the API-Features façade Web service for the SWIM implemented in the Testbed 16.
Filtering capabilities: The API-Features façade for the SWIM messaging services implemented for Testbed 16 lacks the support of advanced filtering as specified in the OGC API-Features Part 3 (OGC 19-079r1). There is a filter parameter declared in the OpenAPI specification of the OGC API — Features Standard. The OGC API — Features service will be enhanced with the support of filtering using Filtering / Common Query Language (CQL2) languages in both text and JSON encoding.
Additional SWIM data: The Testbed 16 API-Feature façade service does not handle any weather data. That activity focused on relaying data and caching features from messages in Aeronautical Information Exchange Model (AIXM) and Flight Information Exchange Model (FIXM). Extensions to AIXM and FIXM were handled in Testbed 16, such as the US Federal Aviation Administration Notice to Air Missions System Event (FNSE) extension and the US National Airspace System (NAS) extension. The API-Features service in Testbed 18 will add weather data provided in the Weather Information Exchange Model (WXXM) or its local extensions.
Updates of OGC API-Features specification: The OGC API-Features Standard has evolved and been revised for several years. The core Standard has a new update officially released as version 1.0.1. The API-Features service in Testbed 18 adopted the latest revision. Related tools and software development kits (SDKs) for OpenAPI have also evolved and advanced recently from community efforts. The implementation of the API-Features service in Testbed 18 leverages the advancements of these SDKs, especially the evolutions in the open source community. The interface parsing and validation will be done with the API-first approach in the Python open source community. This would be a different experience from the previous Java development.
6.2. Internal Architecture
The OGC API-Feature façade for SWIM services was re-implemented and enhanced using the OpenAPI stub generator (python-aiohttp) to support standard search and retrieval of SWIM messaging-accumulated feature data with filtering languages of Filtering / Common Query Language (CQL2) in text format or JSON format. In addition to the filtering capabilities, new data was added with new messaging-harvester for the Integrated Terminal Weather System (ITWS) of the System Wide Information Management (SWIM).
Figure 12 — D101 Component Overview
Filtering language support is primarily based on an extended version of the open source pygeofilter, a Python implementation of cql2-text and cql2-json parsers. There is an example backend translator for Django, SQLAlchemy, GeoPandas, and generic SQL. This implementation forked the open source pygeofilter to extend the SQL backend to work with the PostGIS backend spatiotemporal database.
The overall server-side API-Features library was implemented as a pluggable system to support extension of the server. Two parts are pluggable – backend data providers and frontend output formatters. The backend data providers provide customized interaction with the data source. Different types may require different providers. The current system implemented the data providers with a PostGIS spatiotemporal database. All collection metadata and features are managed with a set of relational entities (tables).
The frontend formatters produce the responses matching the request. In the Testbed 18 implementation, GeoJSON and rendered HTML are supported. The jinja2 templates are used to render HTML output. The rendered HTML page embedded a light-weight JavaScript client to support the interactive retrieval and rendering of results on maps in browsers.
The following figure shows the interaction and data flow between OGC API — Features façade services and SWIM messaging services. In the implementation, the data are first harvested, translated, and cached in a unified spatiotemporal database from the messaging stream of SWIM services. The ingestion of streaming messages is completed with two steps: a Java Messaging Service (JMS) client to receive the messages and a harvester for each data type to load message into the spatiotemporal database. The harvesters implemented in Java in Testbed 16 were reused for Testbed 18, including harvesters for AIM FNS, TFMS, and SFDPS. The SWIMITWS, the harvesters for ingesting ITWS Weather data from SWIM ITWS messages is newly implemented in Testbed 18. The data include 28 feature collections. Each feature collection has a specific handler to complete the necessary translation, reformatting, and loading into the intermediate spatiotemporal database.
OGCAPIUsers-> D101 : API-Features for SWIM features
D101-> SWIMServiceConsumer: Subscription requests
SWIMServiceConsumer -> SWIM: Request for services
SWIM -> SWIMAIMFNS: Subscribe to SWIM AIM FNS service
SWIMAIMFNS -> SWIM: AIM FNS notice to airmen messaging responses
SWIM -> SWIMTFMS: Subscribe to SWIM TFMS service
SWIMTFMS -> SWIM: TFMS traffic flow messaging responses
SWIM -> SWIMSFDPS: Subscribe to SWIM SFDPS service
SWIMSFDPS -> SWIM: SFDPS flight data messaging responses
SWIM -> SWIMITWS: Subscribe to SWIM ITWS service
SWIMITWS -> SWIM: ITWS Weather data messaging responses
SWIM -> SWIMServiceConsumer: Proxy service responses
SWIMServiceConsumer-> D101 : Document-based database indexing
D101->OGCAPIUsers : Resources (GeoJSON, HTML)
Figure 13 — API-Features Data Interaction with Backend SWIM Messaging Services
' hide the spot
hide circle
' avoid problems with angled crows feet
skinparam linetype ortho
entity "Collections" as e01 {
*e1_fid : number <<generated>>
--
*thegeom : geometry
*cid : varchar
table: varchar
title : varchar
description : text
bbox: varchar
temporal: varchar
itemtype: varchar
crs: varchar
links: varchar
}
entity "Collection.Queryables" as e07 {
*e7_id : number <<generated>>
--
*e2_id : number <<FK>>
*cid : varchar <<FK>>
type : varchar
encoding : text
}
entity "CollectionEncoding" as e02 {
*e2_id : number <<generated>>
--
*e1_id : number <<FK>>
*type: varchar
*encoding : text
}
entity "Document" as e03 {
*e3_id : number <<generated>>
--
*file : varchar
*status : number
}
entity "Message" as e04 {
*e4_id : number <<generated>>
--
*e3_id : number <<FK>>
xpath : varchar
type : number
encoding : text
}
entity "Feature" as e05 {
*e5_id : number <<generated>>
--
*thegeom : geometry
*gmlid : varchar
*gmlidtrace : varchar
*featuretype : varchar
*begintime : timestamp
*endtime : timestamp
*isinsttime : Boolean
elevation : double
verticalTop : double
elem: varchar
}
entity "FeatureEncoding" as e06 {
*e6_id : number <<generated>>
--
*e4_id : number <<FK>>
*e5_id : number <<FK>>
xpath : varchar
type : number
encoding : text
}
e01 ||..|{ e02
e05 ||..o{ e06
e06 ||..o{ e04
e03 ||..o{ e04
Figure 14 — Major Entity Relationship Diagram for Managed SWIM Features
Table 1 — Server Endpoints
| Server name | URL for landing page |
|---|---|
| GMU WFS3 | https://cat.csiss.gmu.edu/gmuwfs3/ |
6.3. Feature collections
6.3.1. AIXM Features
The AIXM feature collections provide a façade the ability to access AIXM features available in SWIM services. The following table lists the collections. The direct access point for the collection can be formed by using a collection id as follows.
https://cat.csiss.gmu.edu/gmuwfs3/collections/
For example, the direct access to collection of Apron Element: https://cat.csiss.gmu.edu/gmuwfs3/collections/ApronElement
Table 2 — AIXM Feature Collections from SWIM Data Services
| id | Title | Description |
|---|---|---|
| ApronElement | Apron Element | Parts of a defined apron area. ApronElements may have functional characteristics defined in the ApronElement type. ApronElements may have jetway, fuel, towing, docking, and groundPower services. |
| Airspace | Airspace | A defined three dimensional region of space relevant to air traffic. |
| TaxiwayElement | Taxiway Element | Part of a Taxiway |
| RunwayElement | Runway Element | Runway element may consist of one or more polygons not defined as other portions of the runway class. |
6.3.2. FIXM/Flight Features
The FIXM/Flight feature collections provide a façade with access to FIXM feature and traffic flow information in SWIM services. The traffic flow information is provided by the TFMData Service by the FAA’s Traffic Flow Management System (TFMS). The following table lists the collections. The direct access point for the collection can be formed by using a collection id as follows.
https://cat.csiss.gmu.edu/gmuwfs3/collections/
For example, the direct access to collection of Track Information: https://cat.csiss.gmu.edu/gmuwfs3/collections/trackinformation
Table 3 — FIXM and Traffic Flow Feature Collections from SWIM Data Services
| id | Title | Description |
|---|---|---|
| trackinformation | Track information | Track information of flight. |
| RAPT | RAPT Timeline Data | Route Availability Planning Tool (RAPT) Timeline Data. |
| flightPlanInformation | Flight plan information | Flight plan information. |
| flightPlanAmendmentInformation | Flight plan amendment information | Flight plan amendment information. |
| NasFlightMessage | Flight Message in FIXM with US Extension | FIXM — Flight information with US Extension. Schemas are available at https://www.fixm.aero/download.pl?view=e . |
6.3.3. ITWS Features
ITWS feature collections provide a façade with access to ITWS weather data in SWIM services. The following table lists the collections. The direct access point for the collection can be formed by using collection id as follows. There are currently 28 feature collections available from the SWIM ITWS service.
https://cat.csiss.gmu.edu/gmuwfs3/collections/
For example, the direct access to collection of Precipitation 5nm Product: https://cat.csiss.gmu.edu/gmuwfs3/collections/ITWS9849
Table 4 — ITWS Weather Feature Collections from SWIM Data Services
| id | Title | Description |
|---|---|---|
| ITWS9849 | Precipitation 5nm Product | The 5nm precipitation product shows data from the TDWR surface level scan in National Weather Service (NWS) VIP 6-levels and areas indicated as attenuated data in a gridded format. |
| ITWS9845 | Airport Lightning Warning | The airport lightning warning product is used to notify users that there is the potential for lightning in close proximity to predefined ‘critical regions’ near an airport. The source of the data is the National Lightning Detection Network (NLDN). |
| ITWS9911 | SM SEP 5nm Product | The Storm Motion Storm Extrapolated Position (SM SEP) product shows motion vectors and contours which indicate the predicted future positions of storm cells. One set of data is generated for the current state and for 10 and 20 minutes in the future. |
| ITWS9838 | Tornado Detections Product | The Tornado Detections product includes the location of a tornado as determined by the NEXRAD tornado algorithm. |
| ITWS9903 | Forecast Contour Product | The Forecast Contour product contains four predicted contour lines corresponding to the outer edges of a storm cell for what ITWS calculates to be the position of the outer boundary of the storm cell for 30 and 60 minutes in the future and for both standard and winter forecasts. |
6.4. Filtering Capabilities
The filtering capability of the API-Features supports both cql2-text and cql2-json. The parsing and translating of queries into backend SQL in PostGIS/Postgresql is achieved through a forked, revised open source library, pygeofilter. Two major revisions to pygeofilter are functional mapping and field (property) mapping. The functional mapping list enables the conversion of query functions to equivalent functions in the backend PostGIS/Postgresql database. For example, temporal comparison function “BEFORE” may have to be mapped to “<” in temporal field comparison with proper conversion (i.e., converting from string expression into internal sql timestamp).
The field mapping converts the property to field or extracted field value in a backend spatiotemporal database. The spatiotemporal database in this implementation uses a generic spatiotemporal database. The data for each feature are stored as an encoded blob/text in the encoding field with a corresponding media-type. The primary encoding is GeoJSON (i.e. application/geo+json) besides the native encoding (e.g., application/aixm+xml;version=3.0.1). The field mapping uses the JSON parsing and extraction capabilities of Postgresql database. The following list shows examples of mappings for ITWS weather data.
The parsing and validation of user requests is enabled using API-first approach. The API-first approach assures the consistency of declarations in OpenAPI YAML or JSON. The implementation of OGC API — Features also uses the open-source server stub generator, OpenAPI Generator, to generate stubs for the server-side library. Models are generated using the Python-aiohttp generator specifically for Python programs. The models are linked to the interface parser, connexion. Connexion is a framework that automatically handles HTTP requests based on OpenAPI Specification. The tight links between parameters, models, and responses enable the consistency across-board.
6.4.1. Query
6.4.1.1. Query with cql2-text
The following is an example request on ITWS9849 (Precipitation 5nm Product) — Search airports.
The cql2-text is: itws_sites = ‘CMH’
The URL encoded request is as follows.
6.4.1.2. Query with cql2-json
The cql2-json is as follows.
{
"filter": {
"op": "=",
"args": [
{
"property": "itws_sites"
},
"CMH"
]
}
}
Figure 15
The URL-encoded request is as follows.
6.5. Challenges and Lessons Learned
The following summarizes challenges and lessons learned for the implementation and enablement of filtering capabilities in the API-Features services for SWIM data.
Filtering enablement: The Filtering / Common Query Language (CQL2) standard is evolving. The open-source libraries for supporting the parsing and translation of CQL2 queries into common backend databases queries or query services such as SPARQL and GraphQL are not in sync with the changes to the draft standard. For example, the implementation of the OGC API — Features service in Testbed -18 using Python found pygeofilter as the library that supports both cql2-text and cql2-json.
However, the predicates for spatial and temporal conditions are still in the formats with prefixes s_ and t_ respectively. This leads to the mandatory forking and revision of pygeofilter to be usable for the latest version as defined in OGC 19-079r1 (OGC API – Features – Part 3: Filtering). The open-source library pygeofilter also has limited support on the backend. In Testbed 18, pygeofilter does not support the backend with PostGIS/postgresql. The backend translator must be extensively edited to support the evaluation of CQL2 (either cql2-text or cql2-json) into valid a SQL command for PostGIS/Postgresql.
Filtering capability description: SWIM messaging services provide different features with different information models. These schemas are not easily regenerated automatically from (for example) database schemas. The extraction and encoding of queryables were done manually with extensive study of the original data models, including AIXM, FIXM, WXXM, and specialized FAA extensions. The description of queryables is flexible, but this flexibility may lead the response to be too flexible to be used properly by clients.
The SWIM dataset has pre-defined codes for locations, facilities, and/or status. Some may not be fully expressed with “enum” attributes. For example, the International Air Transport Association’s (IATA) Location Identifier is another standard that is widely used in SWIM messages as attributes. Verifying if the code is valid unless the client checks with a registry service of IATA is not possible. The specification may specify how to describe the queryable in such a case and give examples for describing the queryable.
Filtering performance: Performance is an implementation-specific experience based on balancing the general applicability and the specialization of information models in the backend spatiotemporal database. This Testbed’s implementation considers managing different features generated from SWIM messages. The harvester would create aa feature collection under the same type if the service has not recorded the collection. In order to achieve this flexibility, the implementation populates partially common attributes (e.g., spatial extent, temporal extent) in defined entities and encodes all into a blob field with corresponding media type (e.g., application/geo+json or native media type – application/aixm+xml;version=3.0.1).
Most queryables are only accessible through an encoded blob. In defining the field-mapping, we used JSON query against the blob. These properties/attributes for queries are directly managed by PostGIS/Postgresql. Therefore, the performance improvement with indexing these properties is not supported with this implementation. This setup used in Testbed 18 may work fine for SWIM data services if only a limited number of features are searchable within their valid time range.
API-first approach for implementing services: The Testbed 18 implementation of OGC API — Features uses the API-first approach. In other words, the specification defined in YAML is used at the interface, intermediate internal information models, and response generation. This approach needs a well-defined OpenAPI specification in either JSON or YAML. The implementation starts to form a complete OpenAPI document using the examples in API-Features Core Standard GitHub and adds the filter parameters from the examples in swaggerhub. The API-Features (OGC 17-069r5 and OGC 19-079r) schema does not include paths although these examples include paths. The paths should be an integrated part of the specification.
Having the specification schema repository including more concrete OpenAPI specification examples with fully defined paths at different levels of conformances in YAML or JSON would be helpful. Schemas can be used directly with little configuration changes with the API-first implementation approach. The internal models generated from the YAML specification or JSON have problems in dealing with “oneOf” for parameterization and Geometrycollection GeoJSON object. The generated models need to be revised before the stubs work. This may be an OpenAPI open-source tool issue. The open-source OpenAPI generator, specifically the Python-aiohttp generator, needs to be revised to support the proper generation of models.
The API-first library, connexion, needs to be revised to support proper validation. Alternatively, the specification may avoid this complexity by leaving out the validation of certain property constraints. For example, the BBOX parameter may just be defined as a simple array with the choice of 2-D or 3-D constraints which the connexion not properly validated against the specification. The Geometrycollection may be fully defined in the schema section to enable the generator tool to create inheritable objects in the generated models.
Advanced filtering: SWIM messaging services are streaming data with validated time periods while they are ingested into separate feature collections. For example, the wind information is delivered through three feature collections: itws_9840_start for wind start time, itws_9840_expiration for wind alter information expiration time, and itws_9840_speed for wind speed update. A filter based on wind event would need to be formed as an integrated query against all three feature collections to get the valid wind event information.
Another case is the part-of relationship between SWIM feature collections which also needs an advanced filtering query to get valid information. For example, a Runway feature may consist of several RunwayElements, a Taxiway feature of multiple TaxiwayElements, and an Apron feature of multiple ApronElements. The classes at the highest level (e.g., Runway, Taxiway, Apron) are in different feature collections than their component classes (e.g., RunwayElement, TaxiwayElement, ApronElement). The CQL2 Standard is not designed to work against multiple feature collections even though they may be served in one OGC API — Features service.
Light-weight client for rendering response in browsers: The implementation of OGC API — Features in Testbed 18 uses an embedded leaflet-based, light-weight JavaScript client to render the HTML response in browsers. This light-weight client helps in guiding users to input proper parameters. The jinja2 templates are used to create the templates for rendering results from different feature collections.
7. Filtering Service 1 (D102)
Ecere Corporation provided the D102 Filtering Service as a Web API instance implementing OGC API — Features supporting advanced filtering which can be applied to cascaded services that do not offer these capabilities themselves, such as D100 and D101. Filters can also be pre-defined using extended process execution requests. The service cascades the D100 and D101 service from interactive instruments, Skymantics and George Mason University OGC API — Features endpoints, with proper paging support whether filtering is used or not.
The endpoint is available at: https://maps.gnosis.earth/ogcapi/collections/swim .
7.1. Status Quo
Prior to the testbed, Ecere’s GNOSIS Map Server did not support filtering using the Common Query Language (CQL2) or cascading other map services.
7.2. Internal Architecture
This service is a deployment of the GNOSIS Map Server implementing OGC API Standards and draft specifications. The relevant collections from the deployed endpoint used for this testbed task are found at: (https://maps.gnosis.earth/ogcapi/collections/swim.
The GNOSIS Map Server product is a certified implementation of OGC API — Features — Part 1: Core Standard. The server product also supports extensions including Part 2: Coordinate Reference Systems by reference, as well as the draft Part 3: Filtering specification. The implementation also supports returning filtered results as tiled vector feature data (“vector tiles”) through the OGC API — Tiles — Part 1: Core Standard, including support for the Mapbox Vector Tiles encoding. The D102 service also acts as an OGC API — Features client for cascading to the D100, D101, and the Skymantics services.
Experiments were performed using GeoJSON and Features & Geometry JSON to encode vector features.
7.2.1. Filtering Capabilities
Filtering queries can be specified at request time using Features — Part 3: Filtering, using either the CMSS expression language or the OGC Common Query Language (CQL2) (new capability developed for this initiative). Filters can also be pre-defined using the OGC API — Processes — Part 3: Workflows & Chaining draft specification, supporting the creation of virtual collections.
Support for the OGC API — Features “Search” extension, discussed in more detail in the Filtering Service and Rule Set Engineering Report (ER) (OGC 22-024), is planned for future work. Ecere will be actively following the development of that extension, considering use cases relating to having common basic analytics and search capabilities across different OGC API Standards, such as planned for OGC API — Processes — Part 3: Workflows & Chaining Input and Output modifiers requirements classes, as well as OGC API — Coverages, OGC API — DGGS. Some of these aspects were previously discussed in Testbed 17 — GeoDataCube API and in the May 2022 Space Partition Code Sprint.
7.2.1.1. CQL2
The Ecere implementation supports specifying filters using CQL2, including support for the Basic, Property-Property, and Arithmetic Expressions conformance classes, and the cql2-text encoding. Support for additional conformance classes, as well as the cql2-json encoding, is planned for future work.
Test collections, using Natural Earth data, for some of the CQL2 Abstract Tests are deployed at the following location.
https://maps.gnosis.earth/ogcapi/collections/cql2-test
The following are a couple of example requests using these test collections.
Figure 16 — Single feature returned from CQL2 filter query on test collection
7.2.2. Cascading
Ecere implemented the ability to provide access to cascaded services, while also supporting filtering features returned by those cascaded services, which may not themselves provide filtering support. Pagination challenges specific to this capability were addressed to reflect the fact that not all results returned by the cascaded service will be part of the filtering service responses.
The following link is an example items request to features originating from the cascaded service D100 service provided by interactive instruments.
http://maps.gnosis.earth/ogcapi/collections/swim:d100_airspace:class_c/items
Figure 17 — Single flight plan feature cascaded from Skymantics service
7.2.3. Pre-defining queries based on Processes — Part 3 extension
The OGC API — Features Search extension (described in detail in OGC 22-024 ER) has much in common with the idea of allowing filter, properties (for selection and derived fields/properties), and sortBy to qualify inputs in OGC API — Processes — Part 3: Workflows and Chaining (input and output modifiers requirements classes).
A well-known pass-through process (with support for collection input including filtering) could support an execution request with a syntax equivalent to the Search extension endpoint, similar to how the OGC API — Routes /routes endpoint shares a POST payload syntax with an eventual definition of a well-known routing process. Pre-defined queries could also be parameterized by deploying them as processes, as suggested in the Deployable workflows requirements class of Processes — Part 3: Workflows and Chaining.
The modifiers introduced include the same filter, properties, and sortBy parameters to qualify inputs originating from a data collection or process, whether they are local or remote, as well as outputs resulting from a process. In addition to the ability to select specific fields/properties, the properties parameter can also be used to derive new fields, for example using CQL2 arithmetic expressions.
Processes — Part 3 also defines a Collection output requirements class where the output of the workflow execution is either a dataset landing page (which can contain multiple collections), or a single collection.
For example, the following parameterized query addressing similar use case as the one described further in section 8.3.5 — Support for the Search resources‘s Example 2 taking two parameters, a string enumeration named composition, and a string array named airports, could be expressed in a Part 3-extended OGC API — Processes execution request (using Collection input and Output modifiers) as follows.
Example 1 — Example parameterizable query as a Part 3-extended execution request
{
"process" : "PassThrough",
"inputs" : {
"data" : [ {
"collection" : "apronelement",
"filter": {
"op": "and",
"args": [
{
"op": "=",
"args": [
{"property": "composition"},
{
"$input": {
"composition": { "type": "string", "enum": ["CONC", "..."] }
}
}
]
},
{
"op": "in",
"args": [
{"property": "airport"},
{
"$input": {
"airports": {
"type": "array",
"items": { "type": "string", "enum": ["JFK", "EWR", "LGA", "..."] }
}
}
}
]
}
]
},
"properties": ["geometry", "airport", "type"],
"sortBy": ["airport"]
} ]
}
}
In this example, data is an input defined in the PassThrough well-known process with multiplicity 1..* which is returned as the process output.
The collection property is defined in the Collection input requirements class of OGC API — Processes — Part 3, whereas the filter, properties, and sortby elements specifying a cql2-json filtering expression, selected properties, and an ascending sort order by airport, are defined in Part 3’s Input modifiers requirements class.
These field modifiers can also be used in the context of the Output modifiers requirements class together with the Collection output requirement class or with regular process execution outputs. In the context of a feature collection output, these query parameter building blocks also correspond to the functionality provided by OGC API — Features — Part 3: Filtering, as well as a planned extensions for Coverages and Discrete Global Grid Systems.
This execution request could be deployed as a new process, e.g., ApronFiltering, using the Processes — Part 2: Deploy, Replace, Undeploy extension’s POST operation to /processes together with the Deployable workflows requirements class of Part 3. The resulting process would get listed at /processes with a process description including the input parameters (composition and airports) and could itself be executed by POSTing to /processes/ApronFiltering/execution as follows.
Example 2 — Example execution request of parameterized query deployed as a process
{
"inputs" : {
"composition" : "CONC",
"airports": [ "JFK", "LGA" ]
}
}
Posting this execution request to the execution endpoint without a Prefer: header would result in a synchronous execution that returns the features. With support for the Collection output requirements class, specifying as parameter response=collection would instead return a collection description, as for a GET request to /collections/{collectionId} in Common — Part 2 and Features — Part 1. With response=landingPage, a landing page would be returned for the filtered dataset, allowing retrieval of multiple collections.
Such virtual collections could also be published to a dataset API as a persistent collection, e.g., as /collections/concApronsJFKLGA, using a non-parameterizable execution request as the payload of a POST operation to /collections to create the dynamic collection, with a Processes execution request content media type to be registered, e.g., application/ogcexecreq+json as suggested in Part 3 — Section 14. Media Types.
During this Testbed-18 advanced filtering task, Ecere successfully demonstrated the use of Part 3 extensions to pre-define filtering queries to the cascading service, including the deployment of a sample PassThrough process with support for the filter and properties modifiers, as well as for the Collection Input and Collection output requirements class, with the following limitations:
the filters were expressed using the cql2-text encoding rather than cql2-json;
the input and output were limited to a single collection;
the sortBy modifier (and sorted feature collections in general) remained to be implemented; and
support for parameterized queries and deployable workflows was not yet implemented.
A sample pre-defined query is available from the endpoint:
https://maps.gnosis.earth/ogcapi/processes/PassThrough/execution?response=collection
including the following default pre-defined filter execution request.
Example 3 — Working execution request of filtering query from D100 cascaded collection
{
"process" : "https://maps.gnosis.earth/ogcapi/processes/PassThrough",
"inputs" : {
"data" : [
{
"collection" : "https://maps.gnosis.earth/ogcapi/collections/swim:d100_airports:apronelement",
"filter" : "composition = 'CONC' and airport in ('JFK', 'EWR', 'LGA')",
"properties" : [ "geometry", "airport", "type" ]
}
]
}
}
Figure 18 — Paging through an output collection resulting from the above filter query pre-defined using OGC API - Processes - Part 3
7.2.4. Cross-collections queries
Among advanced filtering capabilities are cross-collections queries, whereby the server is instructed to perform a join between data sources (which could potentially be hosted in two different servers) as a multi-collection query. Although there was no time to implement this capability during the Testbed, some thinking and discussion focused on researching that capability. Whereas the search extension defines a new endpoint at /search, Ecere suggested supporting cross-collection queries for the usual /items endpoint. An example GET request would appear as follows.
GET /collections/apronelement/items?
collections=apron&
properties=*,apron.otherProperty&
filter=associatedApron=apron.id and airport in (JFK, EWR, LGA)&
sortby=airport&
limit=1000
Figure 19
If there are multiple airport aprons matching the same apronelement, this would likely return more entries than available in the apronelement collection. In this case, the items IDs would need to be disambiguated and would not correspond to the typical /collections/apronelement/items/{itemId}.
The following request illustrates a weather use case, where wind speed information could be available either as OGC API — Features or as OGC API — Coverages (in the case of a coverage, winds.geometry would refer to the geometry of each cell).
GET /collections/flightRoutes/items?
collections=https://weather.com/ogcapi/collections/winds&
filter=winds.speed > 100 and s_intersects(geometry, winds.geometry) and departingAirport in (JFK, EWR, LGA)&
sortby=-winds.speed,departingAirport&
limit=1000
Figure 20
If the winds collection supports OGC API — Environmental Data Retrieval (EDR), the flight routes service could use a trajectory request including the flight route geometry to the weather data API, as one potential way to make this efficient. This would avoid the client fetching the full set of weather data, then transmitting it all again to the flight route service, exchanging the full data collection twice, when in fact all that may have been needed is for the first service to send the smaller flight routes geometry to the second service in a trajectory request. In addition, the two services may often be hosted nearby (e.g., both being hosted on Amazon Web Services), while the client is located elsewhere.
7.3. Challenges and Lessons Learned
Building next and prev links that reflect both cascaded and filtered features, while respecting the limits of the original queries, in order to page features properly proved to be an important challenge. Issues with some of the cascaded implementations themselves also made the task more difficult, including the following issues that were filed upstream with pygeoapi:
support for describing object feature properties (#1090);
use of invalid JSON Schema types in queryables resource (#1091); and
500 server error returned when using application/geo+json as the media type for Accept: header (#1108).
The use of complex objects and arrays in feature properties, as well as the occurrence of null geometry and of mixed geometry types by the cascaded services, were other major difficulties that required significant changes and improvements to Ecere’s GNOSIS Map Server implementation.
While implementing support for CQL2, Ecere reviewed the draft CQL2 specification and detected several important issues and suggested additional improvements such as simplifying the grammar to facilitate parser implementations. Several of the issues were resolved during the testbed, including the following.
CQL2 division real or integer? data type dependent? (#711)
/functions — out of scope for CQL2? (in Part 3: Filtering) (#715)
POLYGON\(… in CQL2 examples (#716)
CQL2: Permission 3 (binary comparison operators for time instants) targeting clients? (#719)
CQL2: Clarify that DATE and TIMESTAMP literals are in basic, but not INTERVAL (#720)
CQL2: Requirement 2C is a requirement on the client; 2B “literal value” (#721)
CQL2: Basic Spatial Operators issues (#733)
CQL2: Spatial Operators Issues (#734)
CQL2 (AT): A.12 CQL2 Text A.13 CQL2 JSON (#750)
CQL2: Remove — from list of valid identifier characters? (#766)
It was agreed that the following issues are to be addressed in a future revision of CQL2.
Simplify the cql2-text grammar (future version improvements?) (#705)
CQL2 as an expression language (not only boolean predicates) (#723)
Other issues were still pending completion at the time of writing this report.
CQL2 Escaping (#717)
CQL2: Thoughts on arrays and IN (#718)
CQL2/AT1.6: {p2} and {p3} contribute nothing to result (#768)
Ecere also worked on a UML conceptual model for expressions, operators, and standardized functions as part of the Styles & Symbology SWG activities with the intent for it to be compatible with the CQL2 conformance classes and support a similar modularity, and potentially provide the foundations of an OGC conceptual basis for expressions and filtering.
Figure 21 — Expressions UML Conceptual Model, covering CQL2 capabilities
Figure 22 — Operators UML Conceptual Model, covering CQL2 capabilities
Figure 23 — Standard functions UML Conceptual Model, covering CQL2 capabilities
An important lesson to take away from the Testbed-18 initiative is that while a filtering service can be implemented separately from a distinct OGC API — Features façade service, in many ways this architecture really defeats the purpose of the filtering capabilities. This is a result of a request returning the full unfiltered set of features must still be made to the cascaded service. A more efficient architecture would be required to perform the filtering as close to the data source as possible.
7.4. Recommendations and Future Work
Filter close to the data: In this task, the deliverables architecture called for one component to perform the filtering while another acts as an OGC API façade. The initiative highlighted that this architecture is not ideal, since the filtering service would still need to request the entire dataset prior to performing the filtering. Instead, for a practical application, the filtering should be performed as close to the source of data as possible, within the same component.
Explore the potential of spatial joins: In the context of queries spanning multiple collections, participants briefly discussed the capability to perform queries that not only filter and combine the results from multiple collections, but also perform join operations, including spatiotemporal joins. A typical scenario that is relevant to aviation use cases is to query flight paths which would encounter bad weather. This could be achieved by correlating the geometry of the flight paths with the location of severe weather patterns.
The dataset used for such correlation would not necessarily need to be on the same server. Instead, the client could possibly direct a filtering service (e.g., one with flight paths) to retrieve data from a particular OGC API instance (e.g., one with weather data). It may in fact may faster for the two servers to talk to each other directly, for example if they are both hosted on the same cloud infrastructure, as opposed to the client first retrieving the data, then submitting it as part of its filtering query.
First-Filtering: In addition, the filtering service may be able to perform a first filtering pass that requires retrieving less data from the other service. This capability could be explored in greater depth in a future activity.
8. Filtering Service 2 (D103)
The Filtering Service 2, identified as deliverable 103 or D103, is an OGC Web API instance that supports advanced filtering for services that do not offer these capabilities themselves, such as D100 and D101. D103 was demonstrated by interactive instruments.
8.1. Internal Architecture
The component consists of the following three filter APIs that were deployed.
These filter APIs provide access to the same feature data as the corresponding façade, but in addition also provide support for Part 2: Coordinate Reference Systems by Reference, Part 3: Filtering and Common Query Language (CQL2).
Responses with feature data could be requested in the following representations/formats:
GeoJSON (media type application/geo+json, query parameter f=json)
JSON-FG (media type application/vnd.ogc.fg+json, query parameter f=jsonfg)
HTML (media type text/html, query parameter f=html)
FlatGeobuf (media type application/flatgeobuf, query parameter f=fgb)
8.2. ldproxy
Like Clause 5, the APIs are provided by ldproxy. To support the requirements for advanced filtering of SWIM data, the following enhancements were been implemented in ldproxy during the testbed:
updated to the latest draft version of OGC API — Features — Part 3: Filtering and Common Query Language (CQL2);
support for the CQL2-JSON encoding of the Common Query Language (CQL2);
support for all API building blocks specified in Clauses 4 and 5 of the Testbed-18 Filtering Service and Rule Set Engineering Report (D002); and
support for JSON-FG was updated to the latest draft (version 0.1).
At the time of writing this ER, the updated code is in the branch “multi-queries” of ldproxy and the master branches of XtraPlatform and XtraPlatform Spatial.
8.3. Filtering Capabilities
8.3.1. Support for CQL2 and Part 3 (Filtering) of OGC API Features
All conformance classes of the Common Query Language (CQL2) were supported, except “Accent-insensitive Comparisons,” “Functions,” and “Arithmetic Expressions.” Both the CQL2-Text and CQL2-JSON encodings were supported.
The conformance classes that are supported could be seen at the relative path conformance from the API landing page, for example at https://t18.ldproxy.net/d103_fns/conformance. The CQL2 conformance classes all started with the base URI http://www.opengis.net/spec/cql2/0.0/conf.
Example 1 — Conformance Declaration of the NOTAM Filter API
{
"links": [
{
"rel": "self",
"type": "application/json",
"title": "Dieses Dokument",
"href": "https://t18.ldproxy.net/d103_fns/conformance?f=json"
},
{
"rel": "alternate",
"type": "text/html",
"title": "Dieses Dokument als HTML",
"href": "https://t18.ldproxy.net/d103_fns/conformance?f=html"
}
],
"conformsTo": [
"http://www.opengis.net/spec/cql2/0.0/conf/advanced-comparison-operators",
"http://www.opengis.net/spec/cql2/0.0/conf/array-operators",
"http://www.opengis.net/spec/cql2/0.0/conf/basic-cql2",
"http://www.opengis.net/spec/cql2/0.0/conf/basic-spatial-operators",
"http://www.opengis.net/spec/cql2/0.0/conf/case-insensitive-comparison",
"http://www.opengis.net/spec/cql2/0.0/conf/cql2-json",
"http://www.opengis.net/spec/cql2/0.0/conf/cql2-text",
"http://www.opengis.net/spec/cql2/0.0/conf/property-property",
"http://www.opengis.net/spec/cql2/0.0/conf/spatial-operators",
"http://www.opengis.net/spec/cql2/0.0/conf/temporal-operators",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/core",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/html",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/json",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/oas30",
"http://www.opengis.net/spec/ogcapi-common-2/0.0/conf/collections",
"http://www.opengis.net/spec/ogcapi-common-2/0.0/conf/html",
"http://www.opengis.net/spec/ogcapi-common-2/0.0/conf/json",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/html",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30",
"http://www.opengis.net/spec/ogcapi-features-2/1.0/conf/crs",
"http://www.opengis.net/spec/ogcapi-features-3/0.0/conf/features-filter",
"http://www.opengis.net/spec/ogcapi-features-3/0.0/conf/filter",
"http://www.opengis.net/spec/ogcapi-features-n/0.0/conf/ad-hoc-queries",
"http://www.opengis.net/spec/ogcapi-features-n/0.0/conf/core",
"http://www.opengis.net/spec/ogcapi-features-n/0.0/conf/manage-stored-queries"
]
}
To construct a query with a richer filter than was supported by the D100 Data APIs, clients must inspect the queryable properties of the feature collection (“queryables”). The response is a JSON Schema and every property can be used in a filter expression. Each queryable is described with a title, a description, and a type. Where known, constraints on the range of values are also included. This should support generating meaningful queries as well as clients that dynamically generate a UI for the data. For example, a property with an enum constraint could be represented using a drop-down list.
Example 2 — Queryables of the NOTAM features
{
"title" : "NOTAMs",
"description" : "Notice to Airmen",
"properties" : {
"notam_keyword" : {
"title" : "NOTAM Keyword",
"description" : "Keyword associated with the NOTAM",
"type" : "string",
"enum" : [ "AD", "APRON", "AIRSPACE", "CHART", "COM", "IAP", "NAV", "OBST", "ODP", "ROUTE", "RWY", "SECURITY", "SID", "SPECIAL", "STAR", "SVC", "TWY", "VFP", "CONSTRUCTION", "LTA" ]
},
"notam_function" : {
"title" : "NOTAM Function",
"description" : "Function of the NOTAM (New, Replacement, Cancelled)",
"type" : "string",
"enum" : [ "NOTAMN", "NOTAMR", "NOTAMC" ]
},
"valid_time_begin" : {
"title" : "Valid time (begin)",
"format" : "date-time",
"type" : "string"
},
"valid_time_end" : {
"title" : "Valid time (end)",
"format" : "date-time",
"type" : "string"
},
"text" : {
"title" : "Text",
"description" : "NOTAM condition text.",
"type" : "string"
},
"selection_code" : {
"title" : "Q Code",
"description" : "Q Code value for the NOTAM.",
"type" : "string"
},
"year" : {
"title" : "Year",
"description" : "NOTAM year values per ICAO Annex-15.",
"type" : "string"
},
"number" : {
"title" : "Number",
"description" : "NOTAM number value per ICAO Annex-15.",
"type" : "integer"
},
"scenario" : {
"title" : "Scenario",
"description" : "Identifier of the event scenario used for digital encoding. The mapping can be found in the Event Scenario documents.",
"type" : "string"
},
"affected_fir" : {
"title" : "Flight Information Region",
"description" : "Flight Information Region (FIR) that is impacted by the NOTAM.",
"type" : "string"
},
"location" : {
"title" : "Location designator",
"description" : "NOTAM location designator of the affected airport/heliport or facility.",
"type" : "string"
},
"icao_location" : {
"title" : "ICAO location designator",
"description" : "ICAO location designator, if published.",
"type" : "string"
},
"series" : {
"title" : "Series",
"description" : "NOTAM series value per International Civil Aviation Organization (ICAO) Annex-15.",
"type" : "string"
},
"type" : {
"title" : "Type",
"description" : "NOTAM type value per ICAO Annex-15. Accepted values are: New (N), Replace (R), Cancel (C).",
"type" : "string",
"enum" : [ "N", "R", "C" ]
},
"issued" : {
"title" : "Issued",
"description" : "Issue date/time of the NOTAM.",
"format" : "date-time",
"type" : "string"
},
"traffic" : {
"title" : "Traffic",
"description" : "NOTAM traffic value per ICAO Annex-15.",
"type" : "string"
},
"purpose" : {
"title" : "Purpose",
"description" : "NOTAM purpose value per ICAO Annex-15.",
"type" : "string"
},
"scope" : {
"title" : "Scope",
"description" : "NOTAM scope value per ICAO Annex-15.",
"type" : "string"
},
"minimum_fl" : {
"title" : "Minimum flight level",
"description" : "NOTAM minimum flight level value per ICAO Annex-15.",
"type" : "string"
},
"maximum_fl" : {
"title" : "Maximum flight level",
"description" : "NOTAM maximum flight level value per ICAO Annex-15.",
"type" : "string"
},
"coordinates" : {
"title" : "Coordinates",
"type" : "string"
},
"radius" : {
"title" : "Radius",
"type" : "string"
},
"schedule" : {
"title" : "Schedule",
"description" : "Contains a schedule of activity/outage if the hours of effect are less than 24 hours a day.",
"type" : "string"
},
"lower_limit" : {
"title" : "Lower limit",
"description" : "Specifies the lower height restriction of the NOTAM.",
"type" : "string"
},
"upper_limit" : {
"title" : "Upper limit",
"description" : "Specifies the upper height restriction of the NOTAM.",
"type" : "string"
},
"geometry" : {
"$ref" : "https://geojson.org/schema/Geometry.json"
}
},
"additionalProperties" : false,
"type" : "object",
"$schema" : "https://json-schema.org/draft/2019-09/schema",
"$id" : "https://t18.ldproxy.net/d103_fns/collections/notam/queryables"
}
The code below shows an example of a query with a CQL2-Text filter expression. This request selects events related to runways/taxiways/airspaces along a trajectory in Florida with a buffer that was effective between 6PM and 10PM UTC on July 12th, 2021.
Without the percent encoding:
https://t18.ldproxy.net/d103_fns/collections/notam/items?filter=
S_INTERSECTS(geometry,POLYGON((-79.53576165455738 25.18569533817705, -79.50845450086926 25.091558846018916, -79.50003716505086 24.993903798755063, -79.51083312059717 24.896483025086265, -79.54042748499097 24.803040350710415, -79.58768296340534 24.71716672520052, -79.65078355430599 24.64216222382644, -79.7273043373718 24.580909227512972, -79.81430466182294 24.53576165455737, -79.90844115398109 24.508454500869263, -80.00609620124493 24.50003716505086, -80.10351697491373 24.510833120597162, -80.19695964928958 24.54042748499097, -80.28283327479947 24.58768296340534, -80.35783777617355 24.650783554305985, -80.41909077248702 24.72730433737181, -80.46423834544262 24.81430466182295, -82.46423834544262 29.81430466182295, -82.49154549913074 29.908441153981084, -82.49996283494914 30.006096201244937, -82.48916687940283 30.103516974913735, -82.45957251500903 30.196959649289585, -82.41231703659466 30.28283327479948, -82.34921644569401 30.35783777617356, -82.2726956626282 30.419090772487028, -82.18569533817706 30.46423834544263, -82.09155884601891 30.491545499130737, -81.99390379875507 30.49996283494914, -81.89648302508627 30.489166879402838, -81.80304035071042 30.45957251500903, -81.71716672520051 30.41231703659466, -81.64216222382645 30.349216445694015, -81.58090922751298 30.27269566262819, -81.53576165455738 30.18569533817705, -79.53576165455738 25.18569533817705))) AND
notam_keyword IN ('RWY, 'TWY', 'AIRSPACE') AND
T_INTERSECTS(INTERVAL(valid_time_begin,valid_time_end), INTERVAL('2021-07-12T18:00:00Z','2021-07-12T22:00:00Z'))
Figure 24
8.3.2. Support for property selection
The Filter APIs not only supported filtering, but also supported restricting the feature properties that are returned by implementing the “Property Selection” proposal for OGC API — Features.
To reduce the properties in the response to a subset of all feature properties, these had to be listed in a query parameter properties, e.g., properties=text,notam_keyword,issued.
8.3.3. Support for sorting
In addition to selecting the properties to be included in the response, sorting the features in the response was supported by implementing the “Sorting” requirements class of OGC API Records.
To sort the features in the response, the feature properties that should be used to determine the sort order had to be listed in a query parameter sortby, e.g., sortby=notam_keyword,text.
Similar to the queryables, clients could identify the sortable properties of the feature collection (“sortables”). Again, the response was a JSON Schema, and every property could be used in the “sortby” query parameter.
Example — Sortables of the NOTAM features
The APIs were configured so that the same properties could be used for sorting and in filter expressions.
{
"title" : "NOTAMs",
"description" : "Notice to Airmen",
"properties" : {
"notam_keyword" : {
"title" : "NOTAM Keyword",
"description" : "Keyword associated with the NOTAM",
"type" : "string",
"enum" : [ "AD", "APRON", "AIRSPACE", "CHART", "COM", "IAP", "NAV", "OBST", "ODP", "ROUTE", "RWY", "SECURITY", "SID", "SPECIAL", "STAR", "SVC", "TWY", "VFP", "CONSTRUCTION", "LTA" ]
},
"notam_function" : {
"title" : "NOTAM Function",
"description" : "Function of the NOTAM (New, Replacement, Cancelled)",
"type" : "string",
"enum" : [ "NOTAMN", "NOTAMR", "NOTAMC" ]
},
"valid_time_begin" : {
"title" : "Valid time (begin)",
"format" : "date-time",
"type" : "string"
},
"valid_time_end" : {
"title" : "Valid time (end)",
"format" : "date-time",
"type" : "string"
},
"text" : {
"title" : "Text",
"description" : "NOTAM condition text.",
"type" : "string"
},
"selection_code" : {
"title" : "Q Code",
"description" : "Q Code value for the NOTAM.",
"type" : "string"
},
"year" : {
"title" : "Year",
"description" : "NOTAM year values per ICAO Annex-15.",
"type" : "string"
},
"number" : {
"title" : "Number",
"description" : "NOTAM number value per ICAO Annex-15.",
"type" : "integer"
},
"scenario" : {
"title" : "Scenario",
"description" : "Identifier of the event scenario used for digital encoding. The mapping can be found in the Event Scenario documents.",
"type" : "string"
},
"affected_fir" : {
"title" : "Flight Information Region",
"description" : "Flight Information Region (FIR) that is impacted by the NOTAM.",
"type" : "string"
},
"location" : {
"title" : "Location designator",
"description" : "NOTAM location designator of the affected airport/heliport or facility.",
"type" : "string"
},
"icao_location" : {
"title" : "ICAO location designator",
"description" : "ICAO location designator, if published.",
"type" : "string"
},
"series" : {
"title" : "Series",
"description" : "NOTAM series value per International Civil Aviation Organization (ICAO) Annex-15.",
"type" : "string"
},
"type" : {
"title" : "Type",
"description" : "NOTAM type value per ICAO Annex-15. Accepted values are: New (N), Replace (R), Cancel (C).",
"type" : "string",
"enum" : [ "N", "R", "C" ]
},
"issued" : {
"title" : "Issued",
"description" : "Issue date/time of the NOTAM.",
"format" : "date-time",
"type" : "string"
},
"traffic" : {
"title" : "Traffic",
"description" : "NOTAM traffic value per ICAO Annex-15.",
"type" : "string"
},
"purpose" : {
"title" : "Purpose",
"description" : "NOTAM purpose value per ICAO Annex-15.",
"type" : "string"
},
"scope" : {
"title" : "Scope",
"description" : "NOTAM scope value per ICAO Annex-15.",
"type" : "string"
},
"minimum_fl" : {
"title" : "Minimum flight level",
"description" : "NOTAM minimum flight level value per ICAO Annex-15.",
"type" : "string"
},
"maximum_fl" : {
"title" : "Maximum flight level",
"description" : "NOTAM maximum flight level value per ICAO Annex-15.",
"type" : "string"
},
"coordinates" : {
"title" : "Coordinates",
"type" : "string"
},
"radius" : {
"title" : "Radius",
"type" : "string"
},
"schedule" : {
"title" : "Schedule",
"description" : "Contains a schedule of activity/outage if the hours of effect are less than 24 hours a day.",
"type" : "string"
},
"lower_limit" : {
"title" : "Lower limit",
"description" : "Specifies the lower height restriction of the NOTAM.",
"type" : "string"
},
"upper_limit" : {
"title" : "Upper limit",
"description" : "Specifies the upper height restriction of the NOTAM.",
"type" : "string"
}
},
"additionalProperties" : false,
"type" : "object",
"$schema" : "https://json-schema.org/draft/2019-09/schema",
"$id" : "https://t18.ldproxy.net/d103_fns/collections/notam/sortables"
}
8.3.4. Support for basic filtering according to OGC API Features Core
The Filter APIs also supported the simple filtering mechanisms supported by the corresponding Data API in D100. See here for details and sample requests.
8.3.5. Support for the Search resources
The requirements for advanced filtering of SWIM data extended beyond these existing draft specifications, in particular:
to retrieve features from multiple collections with a single request;
to restrict the query capabilities of a business user; and
to simplify the execution of queries by business users.
The additional API building blocks needed to support these requirements are specified in the Testbed-18 Filtering Service and Rule Set Engineering Report (D002) and have been implemented in the Filter Service D103.
The filtering rules / stored queries used in the scenario were generated by the Developer Client (D105), but as examples the following stored queries were published in the Filter APIs.
Default values were provided for all parameters so that all queries could be executed without providing a parameter.
Example 1 — Stored Query: Concrete apron, runway, and taxiway elements at NYC area airports
A simple query without parameters in the Airports API. It selects all apron, runway, and taxiway elements with concrete at the major NYC area airports.
The response was restricted to the properties “geometry,” “airport,” and “type”.
{
"id": "nyc-concrete-surface-areas",
"title": "Concrete apron, runway and taxiway elements at NYC area airports",
"queries": [
{"collections":["apronelement"]},
{"collections":["runwayelement"]},
{"collections":["taxiwayelement"]}
],
"filter": {
"op": "and",
"args": [
{ "op": "=", "args": [{ "property": "composition" }, "CONC"] },
{ "op": "in", "args": [{ "property": "airport" }, ["JFK", "EWR", "LGA"]] }
]
},
"properties": ["geometry", "airport", "type"],
"limit": 1000
}
Example 2 — Stored Query: Apron, runway, and taxiway elements of a specific type at one or more airports
A similar query in the Airports API, but with two parameters, the type (normal use, parking, shoulder, intersection) and the airports (one or more from a list of airports in the eastern US).
The response was restricted to the properties “geometry,” “airport,” “type,” and “composition”.
{
"id": "surface-areas",
"title": "Apron, runway and taxiway elements of a specific type at one or more airports.",
"queries": [
{ "collections": ["apronelement"] },
{ "collections": ["runwayelement"] },
{ "collections": ["taxiwayelement"] }
],
"filter": {
"op": "and",
"args": [
{
"op": "=",
"args": [
{"property": "type"},
{
"$parameter": {
"type": {
"title": "Type of the apron, runway or taxiway element",
"description": "The following types are distinguished: normal use, parking, shoulder, intersection.",
"type": "string",
"enum": ["NORMAL", "PARKING", "SHOULD", "INTERS"],
"default": "NORMAL"
}
}
}
]
},
{
"op": "in",
"args": [
{"property": "airport"},
{
"$parameter": {
"airports": {
"title": "Airports",
"description": "The 3-letter IATA airport codes or the airports to filter. Specify multiple values as a comma-separated list.",
"type": "array",
"items": {
"type": "string",
"enum": ["JFK", "EWR", "LGA", "BOS", "PIT", "PHL", "DCA", "BWI", "IAD"]
},
"default": ["JFK", "EWR", "LGA"]
}
}
}
]
}
]
},
"properties": ["geometry", "airport", "type", "composition"],
"limit": 1000
}
Example 3 — Stored Query: NOTAMS along the route IAD to JFK
This query in the NOTAM API filtered NOTAMs by location, by keyword, and by time interval. The default values for the parameters were: the route from IAD to JFK with a 50km buffer (location), airspace-related NOTAMs (keyword), and between 15:05 and 16:30 UTC on August 18th, 2022 (time interval).
{
"id": "generic-notam-query",
"title": "filter NOTAMS",
"description": "NOTAMs filtered by location, by keyword and by time interval.",
"collections": ["notam"],
"filter": {
"op": "and",
"args": [
{
"op": "s_intersects",
"args": [
{"property": "geometry"},
{
"$parameter": {
"geometry": {
"title": "NOTAM geometry",
"description": "NOTAMs with a location that intersects the geometry are selected.",
"type": "object",
"required": ["type", "coordinates"],
"properties": { "type": {"type": "string"}, "coordinates": {"type": "array"} },
"default": {
"type": "Polygon",
"coordinates": [
[
[ -74.07517, 41.03269 ],
[ -73.96881, 41.06885 ],
[ -73.85509, 41.08851 ],
[ -73.73847, 41.09087 ],
[ -73.62351, 41.07587 ],
[ -73.51473, 41.04408 ],
[ -73.41637, 40.99675 ],
[ -73.33224, 40.93573 ],
[ -73.26558, 40.86340 ],
[ -73.21892, 40.78257 ],
[ -73.19399, 40.69637 ],
[ -73.19167, 40.60812 ],
[ -73.21195, 40.52123 ],
[ -73.25399, 40.43900 ],
[ -73.31608, 40.36459 ],
[ -73.39583, 40.30082 ],
[ -73.49016, 40.25010 ],
[ -77.14409, 38.57116 ],
[ -77.24494, 38.53223 ],
[ -77.35344, 38.50940 ],
[ -77.46549, 38.50353 ],
[ -77.57685, 38.51484 ],
[ -77.68334, 38.54291 ],
[ -77.78091, 38.58666 ],
[ -77.86586, 38.64445 ],
[ -77.93495, 38.71408 ],
[ -77.98551, 38.79290 ],
[ -78.01555, 38.87792 ],
[ -78.02384, 38.96588 ],
[ -78.01000, 39.05341 ],
[ -77.97446, 39.13715 ],
[ -77.91854, 39.21387 ],
[ -77.84433, 39.28058 ],
[ -77.75467, 39.33469 ],
[ -74.07517, 41.03269 ]
]
]
}
}
}
}
]
},
{
"op": "in",
"args": [
{"property": "notam_keyword"},
{
"$parameter": {
"notamKeywords": {
"title": "NOTAM keywords",
"description": "A list of NOTAM keywords to filter only NOTAMs with the selected keywords. For more information about the NOTAM keywords see https://www.faa.gov/air_traffic/publications/atpubs/notam_html/chap1_section_2.html.",
"type": "array",
"items": {
"type": "string",
"enum": [
"AD", "APRON", "AIRSPACE", "CHART", "COM", "IAP", "NAV", "OBST", "ODP",
"ROUTE", "RWY", "SECURITY", "SID", "SPECIAL", "STAR", "SVC", "TWY", "VFP",
"CONSTRUCTION", "LTA"
]
},
"default": ["AIRSPACE"]
}
}
}
]
},
{
"op": "t_intersects",
"args": [
{
"interval": [ {"property": "valid_time_begin"}, {"property": "valid_time_end"} ]
},
{
"interval": [
{
"$parameter": {
"begin": {
"title": "Begin of time interval",
"description": "UTC timestamp for the begin of time interval written according to RFC 3339. Example: '2022-08-18T15:05:00Z'. Only NOTAMs with temporal validity in the time interval are selected.",
"type": "string",
"format": "date-time",
"default": "2022-08-18T15:05:00Z"
}
}
},
{
"$parameter": {
"end": {
"title": "End of time interval",
"description": "UTC timestamp for the end of time interval written according to RFC 3339. Example: '2022-08-18T16:30:00Z'. Only NOTAMs with temporal validity in the time interval are selected.",
"type": "string",
"format": "date-time",
"default": "2022-08-18T16:30:00Z"
}
}
}
]
}
]
}
]
},
"limit": 10
}
Example 4 — Stored Query: All features of an airport
This query selects all features of an airport.
{
"id": "features-yb-airport",
"title": "All features of an airport",
"queries": [
{ "collections": ["airportheliport" ] },
{ "collections": ["apron" ] },
{ "collections": ["apronelement" ] },
{ "collections": ["guidanceline" ] },
{ "collections": ["runway" ] },
{ "collections": ["runwayblastpad" ] },
{ "collections": ["runwaycentrelinepoint"] },
{ "collections": ["runwayelement" ] },
{ "collections": ["runwaymarking" ] },
{ "collections": ["surveycontrolpoint" ] },
{ "collections": ["taxiholdingposition" ] },
{ "collections": ["taxiway" ] },
{ "collections": ["taxiwayelement" ] },
{ "collections": ["taxiwaymarking" ] },
{ "collections": ["verticalstructure" ] }
],
"filter": {
"op": "=",
"args": [
{"property": "airport"},
{
"$parameter": {
"airport": {
"title": "Airport",
"description": "The 3-letter IATA airport codes of available airports.",
"type": "string",
"enum": ["JFK", "EWR", "LGA", "BOS", "PIT", "PHL", "DCA", "BWI", "IAD"],
"default": "JFK"
}
}
}
]
},
"limit": 10000
}
8.4. HTML Support
Support for HTML in the OGC API – Features Standard is recommended. A server that supports HTML will support browsing the data with a web browser and will enable search engines to crawl and index the dataset.
For these reasons, support for both the HTML and the JSON resources were implemented during Testbed 18. As a result, the site itself already provides a Business Client.
The HTML representation of the Stored Queries resource included an HTML form for each stored query. The form includes:
the descriptive elements (title and descriptions);
a drop-down list for the desired response format for the feature collection; and
For each parameter:
the descriptive elements (title, description and data type); and
a field, drop-down list or multi-selection list depending on the schema of the parameter.