I. Abstract
OGC API Standards specify Web based capabilities that are typically based on polling for collection resource updates (new features, records, items, coverages, maps, etc.). Depending on a collection’s temporal resolution or frequency of updates, an event-driven / Publish-Subscribe architecture provides a timely, efficient and low latency approach for the delivery of data updates or notifications of updates. The OGC API — Environmental Data Retrieval — Part 2: Publish-Subscribe Workflow Standard provides recommendations on applying Publish-Subscribe architectural patterns to implementations of one or more OGC APIs.
II. Keywords
The following are keywords to be used by search engines and document catalogues.
OGC API, Pub/Sub, Publish, Subscribe, Publish-Subscribe, Event driven architecture, Asynchronous, OGC document, OGC
III. Preface
The OGC API — Environmental Data Retrieval — Part 2: Publish-Subscribe Workflow Standard provides:
Requirements for Publish-Subscribe patterns specific to event driven data workflows and
Options for realizing Publish-Subscribe workflow in OGC APIs.
The Standard has been informed by the draft OGC Publish-Subscribe White Paper OGC 20-081, as well as the Discussion paper for Publish-Subscribe workflow in OGC APIs OGC 23-013. The goal of this Standard is to provide a basis for Publish-Subscribe implementation patterns within the OGC API ecosystem.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.
Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.
IV. Security considerations
No security considerations have been made for this Standard.
V. Submitting Organizations
The following organizations submitted this Document to the Open Geospatial Consortium (OGC):
- Meteorological Service of Canada
- UK Met Office
- US National Weather Service
VI. Submitters
All questions regarding this submission should be directed to the editor or the submitters:
| Name | Affiliation |
|---|---|
| Tom Kralidis | Meteorological Service of Canada |
| Chris Little | UK Met Office |
| Mark Burgoyne | UK Met Office |
| Steve Olson | NOAA/NWS |
| Shane Mill | NOAA/NWS |
VII. Acknowledgements
Thanks to the members of the Meteorology and Oceanography Domain Working Group of the OGC as well as Clemens Portele and all contributors of change requests and comments.
1. Scope
The OGC API — Environmental Data Retrieval — Part 2: Publish-Subscribe Workflows Standard defines building blocks that can be assembled to implement Publish-Subscribe workflows (discovery, topic structure, encoding) as part of OGC API — Environmental Data Retrieval — Part 1: Core. A topic structure is the structured information that a publisher makes available to allow subscribers to choose information of interest to them.
This Standard defines a discovery capability that contains a topic structure in support of binding to notifications for data access and retrieval.
This Standard defines a baseline message payload which can contain summary descriptive information in GeoJSON about a given notification for new data events (new granule, new model run, etc.).
2. Conformance
This Standard defines Publish-Subscribe patterns specific to event driven data workflows, as well as options for realizing Publish-Subscribe workflows in implementations of OGC API Standards.
Requirements for two standardization target types are considered:
API integration
Pub/Sub channels, and
Notification message payloads
Conformance with this Standard shall be checked using all the relevant tests specified in Annex A (normative) of this document. The framework, concepts, and methodology for testing, and the criteria to be achieved to claim conformance are specified in the OGC Compliance Testing Policies and Procedures and the OGC Compliance Testing web site.
In order to conform to this Standard, a software implementation shall choose to implement:
Any one of the conformance levels specified in Annex A (normative).
All requirements classes and conformance classes described in this document are owned by the standard(s) identified.
3. Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.
Advanced Message Queueing Protocol (AMQP) v1.0) https://www.oasis-open.org/standard/amqp
MQTT Version 5.0 https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html
AsyncAPI Specification https://www.asyncapi.com/docs/reference/specification/v3.0.0
DRAFT WMO guidance on technical specifications of WIS 2.0 https://wmo-im.github.io/wis2-guide
DRAFT WMO WIS2 Notification Message https://github.com/wmo-im/wis2-notification-message
DRAFT WMO WIS2 Topic Hierarchy https://github.com/wmo-im/wis2-topic-hierarchy
WebSockets https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
IANA Link Relations https://www.iana.org/assignments/link-relations/link-relations.xhtml
Mark Burgoyne, David Blodgett, Charles Heazel, Chris Little: OGC 19-086r5, OGC API — Environmental Data Retrieval Standard. Open Geospatial Consortium (2022). http://www.opengis.net/doc/IS/ogcapi-edr-1/1.0.0.
Clemens Portele, Panagiotis (Peter) A. Vretanos, Charles Heazel: OGC 17-069r4, OGC API — Features — Part 1: Core corrigendum. Open Geospatial Consortium (2022). http://www.opengis.net/doc/IS/ogcapi-features-1/1.0.1.
DRAFT OGC API — Records — Part 1: Core (n.d.). https://docs.ogc.org/DRAFTS/20-004.html
H. Butler, M. Daly, A. Doyle, S. Gillies, S. Hagen, T. Schaub: IETF RFC 7946, The GeoJSON Format. RFC Publisher (2016). https://www.rfc-editor.org/info/rfc7946.
4. Terms, definitions and abbreviated terms
This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.
This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.
For the purposes of this document, the following additional terms and definitions apply.
4.1. Terms and definitions
4.1.1. Broker
Intermediary between Subscribers and other Publishers which have been previously registered with the Broker. The Broker is not the original producer of Messages, but acts as an intermediary, (re-)publishing messages received from other Publishers and decoupling them from their Subscribers.
4.1.2. Collection
A geospatial resource that may be available as one or more sub-resource distributions that conform to one or more OGC API Standards. (OGC 20-024)
4.1.3. Dataset
A collection of data, published or curated by a single agent, and available for access or download in one or more representations. (DCAT)
4.1.4. Distribution
A specific representation of a dataset. A dataset might be available in multiple serializations that may differ in various ways, including natural language, media-type or format, schematic organization, temporal and spatial resolution, level of detail or profiles (which might specify any or all of the above). (DCAT)
4.1.5. Subscriber
An entity that creates a subscription to a Publisher.
4.1.6. Message
A container within which data (such as JSON, XML, binary data, or other content) is transported. Messages may include additional information beyond data, including headers or other metadata used for routing or security purposes.
4.1.7. Channel
A term (string) used to filter messages from a Broker.
4.2. Abbreviated terms
AMQP
Advanced Message Queuing Protocol
AMQPS
Advanced Message Queuing Protocol Secure
API
Application Programming Interface
CORS
Cross-Origin Resource Sharing
HTTP
Hypertext Transfer Protocol
HTTPS
Hypertext Transfer Protocol Secure
IANA
Internet Assigned Numbers Authority
JSON
JavaScript Object Notation
MQP
Message Queuing Protocol
MQTT
Message Queuing Telemetry Transport
MQTTS
Message Queuing Telemetry Transport Secure
OASIS
Organization for the Advancement of Structured Information Standards
OGC
Open Geospatial Consortium
URI
Uniform Resource Identifier
WIS
WMO Information System
WMO
World Meteorological Organization
YAML
YAML Ain’t Markup Language
5. Keywords
6. Conventions
This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, use of XML schema, or special notes regarding how to read the document.
6.1. Identifiers
The normative provisions in this standard are denoted by the URI:
http://www.opengis.net/spec/ogcapi-edr-2/1.0
All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.
6.2. Use of HTTPS
For simplicity, this document in general only refers to the HTTP protocol. This is not meant to exclude the use of HTTPS and simply is a shorthand notation for “HTTP or HTTPS.” In fact, most servers are expected to use HTTPS, not HTTP.
6.3. Link relations
To express relationships between resources, RFC 8288 (Web Linking) is used.
The following registered link relation types [IANA] are used in this document.
collection: The target IRI points to a resource which represents the collection resource for the context IRI.
hub: Refers to a hub that enables registration for notification of updates to the context.
item: Refers to a resource that is a member of the collection represented by this context.
service-desc: Identifies service description for this context that is primarily intended for consumption by machines. API definitions are considered service descriptions.
Each resource representation includes an array of links. Implementations are free to add additional links for all resources provided by the API. For example, an enclosure link could reference a bulk download of a collection. Or a related link on a feature could reference a related feature.
6.4. Examples
Most of the examples provided in this Standard are encoded in JSON. JSON was chosen because it is widely understood by implementers and easy to include in a text document. This convention should NOT be interpreted as a requirement that JSON must be used. Implementers are free to use any format they desire as long as there is a Conformance Class for that format and the deployed API advertises its support for the associated Conformance Class.
6.5. Schemas
AsyncAPI 3.0 Schema objects are used throughout this Standard to define the structure of resources. These schemas are typically represented using YAML encoding. This convention is for the ease of the user. It does not prohibit the use of another schema language or encoding. Nor does it indicate that AsyncAPI 3.0 Schema objects are required. Implementations should use a schema language and encoding appropriate for the format of the resource. Note that for property values in JSON for which null is not explicitly supported/required, server implementations are recommended to drop the property (as opposed to specifying the property with a value of null).
7. Overview
Implementations of OGC API Standards provide Web based capabilities which are typically based on polling for collection resource updates (new features/records items, coverages, maps, etc.). Depending on a collection’s temporal resolution or frequency of updates, an event-driven / Publish-Subscribe architecture provides a timely, efficient and low latency approach for delivery of data resource updates.
The following requirements and recommendations apply to Publish-Subscribe architectural patterns for use with implementations of OGC API Standards.
The Publish-Subscribe architecture assumes reasonable connectivity or favorable DDIL conditions. DDIL refers to Denied, Disrupted, Intermittent, or Limited communications.
Figure 1 — Example of Publish-Subscribe workflow using OGC APIs
8. Requirements Class Publish-Subscribe (Pub/Sub)
8.1. Overview
Requirements class 1: Requirements Class ‘Publish-Subscribe (Pub/Sub)’ | |
|---|---|
| Identifier | http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub |
| Target type | Pub/Sub |
| Conformance class | Conformance class A.1: http://www.opengis.net/spec/ogcapi-edr-2/1.0/conf/pubsub |
| Normative statement | Requirement 1: /req/pubsub/api |
Event-driven workflows provide Publish-Subscribe based capabilities as part of information systems and architectures. The Publish-Subscribe model also provides efficiencies in providing data “as it happens”, thereby preventing potential clients from continuously polling to check on the availability of new data or resources.
The Open Geospatial Consortium (OGC) has conducted significant work on event-based models and architectures. The Publish-Subscribe model results in less network traffic and more timely responses to manage event-based models such as urgent, temporally unpredictable data (examples include, but are not limited to: traffic conditions, weather or hazard warnings, and real-time sensor data).
Building on the OGC Publish-Subscribe Interface Standard OGC 13-131r1, as well as the recommendations put forward in the OGC Pub/Sub White Paper [OGC 20-081] produced as part of OGC Testbed 12, as well as the Discussion paper for Publish-Subscribe workflow in OGC APIs [OGC 23-013], the OGC API — Environmental Data Retrieval — Part 2: Publish-Subscribe Workflow Standard discusses approaches for integrating Publish-Subscribe architecture into the OGC API suite of Standards.
Permission 1 | |
|---|---|
| Identifier | /per/pubsub/protocols |
| A | An implementation of the Publish-Subscribe workflows described in this Standard MAY use the message queueing protocol of their choice and/or based on application requirements. |
8.2. OGC API Considerations
The OGC API building block approach would typically be used for shared components in API implementations in support of a polling workflow. Using HTTP, this means that the client initiates and invokes requests and receives responses from the server. A key concept of the OGC API building blocks architecture is the service endpoint of the URL path specifying a resource and any similar sub-resources, which can be applied for Pub/Sub workflow as follows:
Data producers: Messages are published to a broker, applied to a given channel (example: collections/mycollection).
Broker provisioning: Published messages are sent to subscribers.
Subscribers and data consumers: Messages are received by users subscribed to one or more channels (explicitly or using wildcards or filtering).
The above workflow requires adherence to a structure of information channels, auto-discovery of those channels, as well as processing of generic messages for broad interoperability by all components.
8.2.1. AsyncAPI
Based on research and testing, the Pub/Sub White Paper recommended the use of AsyncAPI. AsyncAPI provides an event-driven equivalent of what is provided by OpenAPI for OGC API Standards (description of protocols, channels, parameters, models, etc.). An implementation of the OGC API landing page requirements class can provide a link to an AsyncAPI document as follows:
{
"rel": "service-desc",
"type": "application/json",
"title": "AsyncAPI document",
"href": "https://example.org/asyncapi"
}
Figure 2 — OGC API landing page example link to an AsyncAPI document
NOTE: the media type for an AsyncAPI document may change in the future as decided by the AsyncAPI community.
Requirement 1 | |
|---|---|
| Identifier | /req/pubsub/api |
| Included in | Requirements class 1: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub |
| A | A landing page SHALL provide a link reference to the description of its Publish-Subscribe capabilities using a link relation of service-desc. |
| B | An API SHALL provide the description of its Publish-Subscribe capabilities using AsyncAPI to describe supported protocols, channels, and message payload descriptions. |
8.2.2. Providing notification metadata as an OGC API endpoint
For Brokers providing notification metadata (as opposed to actual data payloads), an implementation of OGC API Building Blocks can, in parallel, readily provide GeoJSON-based notification messages via an OGC API — Features endpoint. Providing message payloads via an implementation of OGC API Standard(s) provides the additional benefit of querying for past messages over time in case of a lost connection. See Clause 10 for more information.
8.2.3. Providing Pub/Sub links to collection updates
The links array could also provide references to the Pub/Sub capabilities available on the service. A collection link could reference a collection update notification channel.
NOTE: In the OGC API Suite of Standards, a collection is a geospatial resource (such as a dataset) that may be available as one or more sub-resource distributions that conform to one or more OGC API standards. See the OGC API — Common — Part 2: Geospatial Data candidate Standard.
Communicating event driven workflow from a link object is made via the hub link relation. This link relation communicates that the link represents a Publish-Subscribe workflow defined by a Pub/Sub protocol in the href property as well as a channel property. The channel property provides the relevant addressable topic that a client can subscribe to after connecting to a Pub/Sub endpoint. The value and syntax of the channel property is bound to the Pub/Sub protocol identified in the href property.
{
"rel": "hub",
"title": "Data notifications",
"href": "mqtt://example.org:8883",
"channel": "collections"
}
Figure 3 — Example of OGC API Pub/Sub link to new collection notifications
8.2.4. Providing Pub/Sub links to collection item notifications
An items link could reference a data payload channel:
An OGC API — Features example
{
"rel": "hub",
"title": "Data notifications",
"href": "mqtt://example.org:8883",
"channel": "collections/surface-weather-observations"
}
Figure 4 — Example of OGC API - Features linking to a data payload channel
An OGC API — EDR example
{
"rel": "hub",
"title": "Data notifications",
"href": "mqtt://example.org:8883",
"channel": "collections/surface-weather-observations/items"
}
Figure 5 — Example of OGC API - EDR linking to a data payload channel
Permission 2 | |
|---|---|
| Identifier | /per/pubsub/links |
| A | A collection resource MAY provide a link reference to a Publish-Subscribe server from an OGC API implementation endpoint when Publish-Subscribe capabilities exist related to the collection service endpoint. |
| B | A Publish-Subscribe collection link reference MAY provide a channel property to allow for granular subscription. |
9. Requirements Class Publish-Subscribe (Pub/Sub) Channels
9.1. Overview
9.1.1. Channels
The OGC API service endpoint specified by a URL path of resources and sub-resources can be used in parallel as a channel description when the data publisher wishes to provide Pub/Sub capability for resources normally available via an OGC API implementations instance in the same way. Below are examples of service endpoints or resources normally available via HTTP, and how they can be re-used as topics for Pub/Sub workflow:
/collections: Notifies Subscribers whenever there is a change to the /collections resource (for example, addition of a new collection). The message payload would be collection metadata as defined in the OGC API — Common — Part 2: Geospatial Data candidate Standard, or a message referencing the collection metadata.
/collections/{collectionId}: Notifies Subscribers whenever there is an update to a single collection resource (for example, spatial or temporal extents, new items, etc.). The message payload would be defined by either the resource model of the given collection (items, etc.), or a notification message of metadata referencing the collection with the relevant change.
For example, users could use a subscription to metadata records, which are usually small compared to the source data, and are therefore more transportable. This informs and notifies the user of changes prior to requesting the possibly large source data, especially when bandwidth is at a premium.
Using the OGC API service endpoints of the URL path of resource and sub-resources provides the key benefit that developers implementing OGC API Standards do not need to learn a different, additional approach or resource path for Pub/Sub (same content, additional interface).
10. Requirements Class Publish-Subscribe (Pub/Sub) Notification Message Payloads
10.1. Overview
Requirements class 2: Requirements Class ‘Publish-Subscribe (Pub/Sub) Notification Message Payloads’ | |
|---|---|
| Identifier | http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub-notification-message-payload |
| Target type | Pub/Sub |
| Conformance class | Conformance class A.2: http://www.opengis.net/spec/ogcapi-edr-2/1.0/conf/pubsub-notification-message-payload |
| Prerequisite | /req/pubsub |
| Normative statements | Requirement 2: /req/pubsub-notification-message-payload/geojson Requirement 3: /req/pubsub-notification-message-payload/id Requirement 5: /req/pubsub-notification-message-payload/operation Requirement 4: /req/pubsub-notification-message-payload/pubtime |
A key component of Pub/Sub workflows is the message payload. Once a client subscribes to one or more channels from a given Pub/Sub server, notifications messages are sent using a given representation or encoding. Notification messages can be issued using any encoding that is deemed suitable by a given publisher.
While the Publish-Subscribe (Pub/Sub) Requirements Class recommends a machine-readable message payload, the Notification Message Payload Requirements Class provides further requirements for interoperability of message payloads as part of an OGC API implementation ecosystem.
10.1.1. GeoJSON
GeoJSON can be used for geospatial data to improve interoperability, but it specifies that geospatial coordinate data must only use the WGS84 coordinate reference system. GeoJSON is defined in the standard IETF RFC 7946.
Requirement 2 | |
|---|---|
| Identifier | /req/pubsub-notification-message-payload/geojson |
| Included in | Requirements class 2: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub-notification-message-payload |
| A | A Pub/Sub notification message encoding SHALL be compliant to IETF RFC 7946 GeoJSON. |
10.1.2. Identifier
A universally unique identifier of the message using the Universally Unique IDentifier (UUID) standard (RFC 4122). The identifier is generated by the originator of the message.
It remains the same throughout the lifetime of the message. The identifier is valuable for “replay” feed services, where a Pub/Sub client may have gone offline for a period of time. Note that multiple messages may communicate updates on the same resource, in which case the message payload can provide a resource identifier in another property.
"id": "31e9d66a-cd83-4174-9429-b932f1abe1be"
Figure 6 — Example id property
Requirement 3 | |
|---|---|
| Identifier | /req/pubsub-notification-message-payload/id |
| Included in | Requirements class 2: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub-notification-message-payload |
| A | A Pub/Sub notification message SHALL provide an id property as a globally unique identifier for the message. |
Recommendation 1 | |
|---|---|
| Identifier | /rec/pubsub-notification-message-payload/id |
| A | For message payloads that provide notification metadata about a resource publication, a Pub/Sub notification message id property SHOULD use a UUID. |
10.1.2.1. pubtime
The pubtime property identifies the date/time of when the message was posted/published. datetime is published as specified in RFC 3339 Clause 5.6 in the UTC timezone (Z). The publication date/time is critical for subscribers to prevent message loss in providing awareness of how far behind the publisher they may be.
"properties": {
...
"pubtime": "2022-03-20T04:50:18.314854383Z"
...
}
Figure 7 — Example pubtime property
Requirement 4 | |
|---|---|
| Identifier | /req/pubsub-notification-message-payload/pubtime |
| Included in | Requirements class 2: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub-notification-message-payload |
| A | A Pub/Sub notification message SHALL provide a properties.pubtime property in RFC 3339 format. |
10.1.2.2. operation
The operation property indicates the stage of the lifecycle for the resource described in the notification, and can be used to notify users that a resource has been created, updated or deleted. If not specified, the default value is create. Other allowed values are update and delete.
"properties": {
...
"operation": "create",
...
}
Figure 8 — Example operation property for a creation
"properties": {
...
"operation": "update",
...
}
Figure 9 — Example operation property for an update
"properties": {
...
"operation": "delete",
...
}
Figure 10 — Example operation property for a deletion
Requirement 5 | |
|---|---|
| Identifier | /req/pubsub-notification-message-payload/operation |
| Included in | Requirements class 2: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub-notification-message-payload |
| A | A Pub/Sub Notification Message SHALL provide the properties.operation property to indicate if a resource has been created, updated or deleted. |
Annex A
(normative)
Conformance Class Abstract Test Suite
A.1. Conformance Class Publish-Subscribe (Pub/Sub)
Conformance class A.1 | |
|---|---|
| Identifier | http://www.opengis.net/spec/ogcapi-edr-2/1.0/conf/pubsub |
| Requirements class | Requirements class 1: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub |
| Target Type | Pub/Sub |
| Conformance test | Abstract test A.1: /conf/pubsub/api |
Abstract test A.1 | |
|---|---|
| Identifier | /conf/pubsub/api |
| Requirement | Requirement 1: /req/pubsub/api |
| Test purpose | Validate that an implementation of OGC API — EDR provides AsyncAPI capabilities. |
| Test method |
|
A.2. Conformance Class Publish-Subscribe (Pub/Sub) Message Payloads
Conformance class A.2 | |
|---|---|
| Identifier | http://www.opengis.net/spec/ogcapi-edr-2/1.0/conf/pubsub-notification-message-payload |
| Requirements class | Requirements class 2: http://www.opengis.net/spec/ogcapi-edr-2/1.0/req/pubsub-notification-message-payload |
| Conformance tests | Abstract test A.2: /conf/pubsub-notification-message-payload/geojson Abstract test A.3: /conf/pubsub-notification-message-payload/id Abstract test A.4: /conf/pubsub-notification-message-payload/operation Abstract test A.5: /conf/pubsub-notification-message-payload/pubtime |
Abstract test A.2 | |
|---|---|
| Identifier | /conf/pubsub-notification-message-payload/geojson |
| Requirement | Requirement 2: /req/pubsub-notification-message-payload/geojson |
| Test purpose | Validate that a notification message is a valid GeoJSON document. |
| Test method |
|
Abstract test A.3 | |
|---|---|
| Identifier | /conf/pubsub-notification-message-payload/id |
| Requirement | Requirement 3: /req/pubsub-notification-message-payload/id |
| Test purpose | Validate that a notification message provides an id property that is a UUID. |
| Test method |
|
Abstract test A.4 | |
|---|---|
| Identifier | /conf/pubsub-notification-message-payload/operation |
| Requirement | Requirement 5: /req/pubsub-notification-message-payload/operation |
| Test purpose | Validate that a notification message provides a properties.operation property. |
| Test method |
|
Abstract test A.5 | |
|---|---|
| Identifier | /conf/pubsub-notification-message-payload/pubtime |
| Requirement | Requirement 4: /req/pubsub-notification-message-payload/pubtime |
| Test purpose | Validate that a notification message provides a properties.pubtime property. |
| Test method |
|
Annex B
(informative)
Examples
B.1. Pub/Sub API Description Example
The API is described using the AsyncAPI 3.0.0 specification and an example response can be found below:
Example 1
asyncapi: '3.0.0'
info:
title: 'Example API conforming to version 1.0.0 of the OGC API - Environmental Data Retrieval - Part 2: Publish-Subscribe Workflow Standard'
version: 1.0.0
description: "AsyncAPI description of an example API conforming to version 1.0.0 of the OGC API - Environmental Data Retrieval - Part 2: Publish-Subscribe Workflow Standard. \n\n Copyright (c) 2024 Open Geospatial Consortium. To obtain additional rights of use, visit http://www.ogc.org/legal/ \n\n The OGC API - Environmental Data Retrieval - Part 2: Publish-Subscribe Workflow Standard provides recommendations on applying Publish-Subscribe architectural patterns to implementations of one or more OGC API Standards."
contact:
name: Open Geospatial Consortium (OGC)
url: https://www.ogc.org/contacts
email: standards-team@ogc.org
license:
name: OGC license
url: http://www.ogc.org/legal/
servers:
mqtt_prod:
host: example.org
protocol: mqtt
description: MQTT endpoint
security:
- type: userPassword
defaultContentType: application/json
channels:
notify-collections:
address: collections
messages:
collection_msg:
description: collection updated notification
payload:
$ref: '#/components/schemas/collection_msg'
notify-collections-wthr-stn:
address: collections/wthr_st
messages:
collection_msg:
description: collection updated notification
payload:
$ref: '#/components/schemas/collection_msg'
notify-collections-stream-gage:
address: collections/stream_gage
messages:
collection_msg:
description: collection updated notification
payload:
$ref: '#/components/schemas/collection_msg'
notify-collections-wthr_stn-items:
address: collections/wthr_stn/items
messages:
wthr_stn_msg:
description: An observation formatted as GeoJSON
payload:
$ref: '#/components/schemas/wthr_stn_msg'
notify-collections-stream_gage-items:
address: collections/stream_gage/items
messages:
stream_gage_msg:
description: Monitoring station data formatted as GeoJSON
payload:
$ref: '#/components/schemas/stream_gage_msg'
operations:
notify-collections:
action: receive
channel:
$ref: '#/channels/notify-collections'
notify-collections-wthr-stn:
action: receive
channel:
$ref: '#/channels/notify-collections-wthr-stn'
notify-collections-stream-gage:
action: receive
channel:
$ref: '#/channels/notify-collections-stream-gage'
notify-collections-wthr_stn-items:
action: receive
channel:
$ref: '#/channels/notify-collections-wthr_stn-items'
notify-collections-stream_gage-items:
action: receive
channel:
$ref: '#/channels/notify-collections-stream_gage-items'
components:
schemas:
collection_msg:
type: object
required:
- id
- href
properties:
id:
type: string
description: collection name
time:
type: string
format: date-time
description: time collection changed
href:
type: string
format: uri
description: URL of the changed collection
wthr_stn_msg:
type: object
additionalProperties: false
properties:
id:
type: string
type:
type: string
geometry:
type: object
properties:
type:
type: string
coordinates:
type: array
items:
type: number
format: float
properties:
type: object
properties:
time:
type: string
format: date-time
id:
type: string
wind_direction:
type: number
format: float
wind_speed:
type: number
format: float
wind_gust:
type: number
format: float
visibility:
type: number
format: float
air_temperature:
type: number
format: float
dew_point:
type: number
format: float
mean_sea_level_pressure:
type: number
format: float
stream_gage_msg:
type: object
additionalProperties: false
properties:
id:
type: string
type:
type: string
geometry:
type: object
properties:
type:
type: string
coordinates:
type: array
items:
type: number
format: float
links:
type: array
items:
type: object
properties:
rel:
type: string
type:
type: string
title:
type: string
href:
type: string
format: uri
properties:
type: object
properties:
datetime:
type: string
format: date-time
label:
type: string
parametername:
type: array
items:
type: string
edrqueryendpoint:
type: string
format: uri
Breaking down into the components:
Example 2
asyncapi: '3.0.0'
info:
title: AsyncAPI demo
version: '0.0.1'
description: |
AsyncAPI description of the proposed Pub/Sub functionality
contact:
name: Contact Name
email: you@example.org
The asyncapi field indicates you use the AsyncAPI version 3.0.
The info field holds information about the API, such as its name, version, description, and license.
Example 3
servers:
mqtt_prod:
host: example.org
protocol: mqtt
protocolVersion: 5.0
description: MQTT endpoint
security:
- user-password: []
Each server object provides the following fields:
host: the server hostname and port
protocol: Pub/Sub protocol supported by the server
protocolVersion: version of the Pub/Sub protocol supported by the server
description: string describing the host
security: reference to supported authentication types
Example 4
servers:
mqtt:
host: example.org
protocol: mqtt
protocolVersion: 5.0
description: MQTT endpoint
variables:
port:
enum:
- 1883
default: 1883
amqp:
host: example.org
protocol: amqp
protocolVersion: 1.0.0
description: AMQP endpoint
variables:
port:
enum:
- 5672
default: 5672
Different Pub/Sub protocols are supported as additional server objects, and can be defined accordingly.
Example 5
channels:
notify-collections:
address: collections
message:
$ref: '#/components/messages/collection_msg'
notify-collections/wthr_stn:
address: collections/wthr_stn
messages:
$ref: '#/components/messages/collection_msg'
notify-collections-stream_gage:
address: collections/stream_gage
messages:
$ref: '#/components/messages/collection_msg'
notify-collections-wthr_stn-items:
address: collections/wthr_stn/items
messages:
$ref: '#/components/messages/wthr_stn_msg'
collections-stream_gage-items:
address: collections/stream_gage/items
messages:
$ref: '#/components/messages/stream_gage_msg'
The channels section lists the events a user can subscribe to and can provide a schema for the associated message payloads.
In the example the following events can be subscribed to:
collections
collections/wthr_stn
collections/stream_gage
collections/wthr_stn/items
collections/stream_gage/items
Example 6
operations:
notify-collections:
action: receive
channel:
$ref: '#/channels/notify-collections'
notify-collections-wthr-stn:
action: receive
channel:
$ref: '#/channels/notify-collections-wthr-stn'
notify-collections-stream-gage:
action: receive
channel:
$ref: '#/channels/notify-collections-stream_gage'
notify-collections-wthr_stn-items:
action: receive
channel:
$ref: '#/channels/notify-collections-wthr_stn-items'
notify-collections-stream_gage-items:
action: receive
channel:
$ref: '#/channels/notify-collections-stream_gage-items'
The operations section lists the required operations and their send and receive capabilities.
Example 7
components:
As in the OpenAPI specification, the components section is used to define reusable objects for different aspects of the AsyncAPI specification.
Annex C
(informative)
Pub/Sub Message Payload Examples
C.1. Pub/Sub Message Payload Example
The World Meteorological Organization (WMO) is a specialized agency of the United Nations responsible for promoting international cooperation on meteorological, climatological, hydrological, and related environmental services, to improve well-being of all. The WMO WIS2 standard notification message format ensures that the WIS2 ecosystem (data publisher, data user, and global services) is a robust, effective, and unified exchange platform for weather, climate, and water data. The message provides notification metadata about the availability of a new data granule. The message is encoded using a GeoJSON object, and provides detailed information on the data notification (associated datetime of the granule, publishing datetime, integrity), as well as access to the data via a link object or inline content (useful for encoding small messages). Geometry is required (given GeoJSON requirements), however geometry can be expressed with a null value when generating the geometry in the message is not possible, practical or timely for data publishers. To support extensibility, additional properties are also valid (given the default definition in JSON Schema).
Using a GeoJSON object as the message payload supports broad interoperability given the large ecosystem of tooling (decoders, encoders) supporting the same approach. An example web application demonstrating the ease of integration can be found at https://kralidis.ca/tmp/wis2-data-notifications.html.
An example WIS2 Notification Message can be found below, extending the OGC API — Pub/Sub Notification Message Requirements with domain specific properties as required:
Example
{
"id": "31e9d66a-cd83-4174-9429-b932f1abe1be",
"version": "v04",
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [
6.146255135536194,
46.223296618227444
]
},
"properties": {
"pubtime": "2022-03-20T04:50:18.314854383Z",
"operation": "create",
"datetime": "2022-03-20T04:45:00Z",
"integrity": {
"method": "sha512",
"value": "A2KNxvks...S8qfSCw=="
},
"data_id": "dataset/123/data-granule/UANT01_CWAO_200445___15103.bufr4",
"metadata_id": "urn:x-wmo:md:can:eccc-msc:observations.swob",
"content": {
"encoding": "utf-8",
"value": "encoded bytes from the file",
"size": 457
}
},
"links": [
{
"href": "https://example.org/data/4Pubsub/92c557ef-d28e-4713-91af-2e2e7be6f8ab.bufr4",
"rel": "canonical",
"type": "application/x-bufr"
},
{
"href": "https://example.org/oapi/collections/my-dataset/items/my-data-granule",
"rel": "item",
"type": "application/json"
}
]
}
C.2. Pub/Sub Message Payload Schema
Example
$schema: 'https://json-schema.org/draft/2020-12/schema'
$id: 'https://schemas.opengis.net/ogcapi/edr/part2/1.0/openapi/schemas/pubsub-message-payload-schema.yaml'
title: OGC API - Pub/Sub message payload definition
description: OGC API - Pub/Sub message payload definition
required:
- id
- type
- geometry
- properties
properties:
id:
$ref: 'https://schemas.opengis.net/ogcapi/features/part1/1.0/openapi/schemas/featureGeoJSON.yaml#/properties/id'
type:
$ref: 'https://schemas.opengis.net/ogcapi/features/part1/1.0/openapi/schemas/featureGeoJSON.yaml#/properties/type'
geometry:
$ref: 'https://schemas.opengis.net/ogcapi/features/part1/1.0/openapi/schemas/featureGeoJSON.yaml#/properties/geometry'
properties:
type: object
required:
- pubtime
properties:
resourceId:
type: string
description: |
Identifies a resource identifier that may have multiple message notifications on
its state or lifecycle over time.
pubtime:
type: string
format: date-time
description: |
Identifies the date/time of when the file was posted/published, in RFC 3339 format.
The publication date/time is critical for subscribers to prevent message loss by knowing
their lag (how far behind the publisher they are).
operation:
type: string
description: the event associated with the lifecycle of a resource.
enum:
- create
- update
- delete
default: create
links:
type: array
items:
$ref: 'https://schemas.opengis.net/ogcapi/common/part1/1.0/openapi/schemas/link.yaml'
Annex D
(informative)
Use cases
D.1. Earth System Prediction model run and data granules notification
A given numerical weather prediction system produces a weather forecast as part of a model run. A model run typically has associated forecast hours. Each forecast hour makes available one or many weather elements at different pressure levels of the atmosphere.
For example, Canada’s Global Deterministic Prediction System (GDPS) produces two model runs per day, providing forecast of numerous weather elements for 33 pressure levels, at a resolution of 15 kilometres.1.
A Pub/Sub workflow can be applied to an NWP system where:
notifications are sent as individual weather elements (data granules) become available
notifications are sent once the model run generation is complete, or “fully qualified”
As a result, Pub/Sub would eliminate the need for continuous polling while a model run is in progress.
Annex E
(informative)
Revision History
Table E.1
| Date | Release | Editor | Primary clauses modified | Description |
|---|---|---|---|---|
| 2023-08-28 | 0.1 | T. Kralidis | all | bootstrap |
| 2024-01-10 | 0.2 | C. Little | all | editorial consistency |
| 2024-02-16 | 0.3 | C. Little | all | workflow consistency |
| 2024-05-07 | 0.4 | T. Kralidis | all | address review comments |
Bibliography
[1] Tom Kralidis, Mark Burgoyne, Steve Olson, Shane Mill: OGC 23-013, Discussion paper for Publish-Subscribe workflow in OGC APIs. Open Geospatial Consortium (2023). http://www.opengis.net/doc/discussion-paper/ogcapi-pubsub/1.0.
[2] Aaron Braeckel, Lorenzo Bigagli, Johannes Echterhoff: OGC 13-131r1, OGC® Publish/Subscribe Interface Standard 1.0 — Core. Open Geospatial Consortium (2016). http://www.opengis.net/doc/IS/pubsub-core/1.0.0.
[3] OGC Publish-Subscribe White Paper (2020)