OGC API Hackathon 2019 Engineering Report

All questions regarding this document should be directed to the editor or the contributors:

Contacts

The following table lists the Hackathon Team Leads and their affiliated working groups.

Team Leads

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

API Application Programming Interface CRS Coordinate Reference System CSW Catalogue Service for the Web GML Geography Markup Language HTML Hypertext Markup Language HTTP Hypertext Transfer Protocol JSON JavaScript Object Notation WCS Web Coverage Service WFS Web Feature Service WMS Web Map Service WMTS Web Map Tile Service WPS Web Processing Service OWS OGC Web Services REST Representational State Transfer XML Extensible Markup Language

The challenge of the Hackathon was to define and test common elements from Coverages, Map Tiles, and Processing standards work using foundational material from the Features work. The magnitude of this challenge was reflected by the fact that the OGC API specifications for Coverages, Map Tiles, and Processing were all at different stages of development. Therefore, the Hackathon had to advance the development of all of the specifications to a stage where common elements across all of the specifications could be identified.

To facilitate the development of the OGC API specifications, the scenario presented in this section was provided as a reference for the teams [3]. The scenario is based on flood risk management and is motivated by recent events such as the 2018 floods that affected parts of Europe (including the United Kingdom, Italy, France, Spain and Portugal) [4] and the 2019 floods that affected parts of the United States [5]. The scenario draws from the OGC’s Disasters Interoperability Concept Development Study (CDS) which assessed geospatial Web services across the disaster domain, defining the core components of National Spatial Data Infrastructure (SDI) architecture for disasters (Disasters SDI), and defining use cases and scenarios for future implementations as part of a follow-on pilot phase [6].

Risk mitigation is one of the phases in the ‘life cycle’ of disaster management [6], which includes the steps shown in Figure 1. Mitigation refers to taking sustained actions to minimize or completely eliminate the long-term risk from hazards and their effects on individuals and property. Preparedness refers to building the emergency management capabilities to respond effectively to any hazard, as well as to recover from the hazard. Response refers to conducting emergency operations that reduce the hazard to acceptable levels (or eliminate it entirely) in order to save lives, through evacuation of potential victims, and provision of food, water, medical care, and shelter to those affected by the disaster. Recovery refers to the rebuilding of communities that have been affected by a disaster so that those communities, as well as their governments, can return to normality and function on their own. A more detailed discussion on disaster management can be found in the OGC Development of Disaster Spatial Data Infrastructures for Disaster Resilience Engineering Report [6].

As part of Government flood risk management policy, Local Authorities have to carry out a preliminary flood risk analysis. Using satellite imagery, flood risk data, along with asset information, vulnerable property information and topographic data, Local Authorities carry out analysis to improve resilience and promote a more efficient use of resource.

A Local Authority is tasked with identifying at-risk residential properties in order to assist in flood prevention and amelioration. By carrying out this task, the Local Authority aims to reduce the number of residential properties affected by floods, as well as to decrease the economic and social costs associated with such devastating events. The Geospatial Specialists at the Local Authority embark on the steps presented in Table 1 in order to carry out the task.

The illustration in Figure 2 shows the Area of Interest (AOI) that was selected to facilitate prototyping, demonstration and briefings. The AOI covered the region of Carmarthenshire, Wales and focused on the town of Carmarthen. The region was the site of significant flooding in October 2018 and hence provided an appropriate location for the flood-based scenario adopted for the Hackathon.

Whereas the Time-Of-Interest (TOI) was October 2018, the AOI had the polygonal bounds in World Geodetic System 1984 (WGS84) longitude-latitude coordinates:

The Hackathon was sponsored by the European Space Agency (ESA) and Ordnance Survey. Table 2 lists organizations that participated in the Hackathon.

The OGC API - Common specification documents the set of common practices and shared requirements that have emerged from the development of Resource Oriented Architectures and Web APIs within the OGC [15]. The specification serves as a common foundation upon which all OGC APIs will be built. As such, the OGC API - Common standard serves as the "OWS Common" standard for resource-oriented OGC APIs. Consistent with the architecture of the Web, this specification uses a resource architecture that conforms to principles of Representational State Transfer (REST). The specification establishes a common pattern that is based on OpenAPI.

In addition to identifying core resources, the OGC API - Common specification goes on to specify HTTP status codes that may be supported by an OGC API, as well as how to handle a coordinate reference system (CRS), web caching, and encodings. The specification also describes how to handle common parameters such as bounding boxes and date-time constraints.

The following subsection presents a summary of the core resources.

The OGC API - Features specification offers the capability to create, modify, and query spatial data on the Web [16] and specifies requirements and recommendations for APIs that want to follow a standard way of sharing feature data. The specification is a multi-part document. The Core part of the specification describes the mandatory capabilities that every implementing service has to support and is restricted to read-access to spatial data. Additional capabilities that address specific needs will be specified in additional parts. Envisaged future capabilities include, for example, support for creating and modifying data, more complex data models, richer queries, and additional coordinate reference systems. This specification builds on the Web Feature Service (WFS) standard and has previously been referred to as WFS 3.0.

Note that the first part of the specification, OGC API - Features - Part 1: Core, has since the hackathon been approved and published as an OGC standard.

The OGC API - Processes specification defines how a client application can request the execution of a process, how the inputs to that process can be provided, and how the output from the process is handled [17]. The specification allows for the wrapping of computational tasks into an executable process that can be invoked by a client application. Examples of computational processes that can be supported by implementations of this specification include raster algebra, geometry buffering, constructive area geometry, routing and several others. This specification builds on the Web Processing Service (WPS) standard.

The OGC API - Map Tiles specification defines an OGC standard for a Web Map Tile API that can serve map tiles of spatially referenced data using tile images with predefined content, extent, and resolution [18]. The specification describes the discovery and query operations of an API that provides access to Map Tiles in a manner independent of the underlying data store. The discovery operations allow the API to be interrogated to determine its capabilities and retrieve metadata about the organization and distribution of tiles. The query operations allow tiles to be retrieved from the underlying data store based upon simple selection criteria, defined by the client. This specification builds on the Web Map Tile Service (WMTS) standard.

Note that this API specification has, at different times, been referred to as the OGC API - Maps and Tiles specification. The different name is a reflection of the on-going discussion about whether the specification should be split up into separate building blocks; one for maps and the other for tiles. A summary of the discussion can be found in the section OGC API - Map Tiles.

The OGC API - Coverages specification defines a Web API for accessing coverages that are modeled according to the Coverage Implementation Schema (CIS) 1.1 [19]. Coverages are represented by some binary or ASCII serialization, specified by some data (en­coding) format. Arguably the most popular type of coverage is that of a gridded coverage. Gridded coverages have a grid as their domain set describing the direct positions in multi-dimensional coordinate space, depending on the type of grid. Satellite imagery is typically modeled as a gridded coverage, for example. The OGC API - Coverages specification builds on the Web Coverage Service (WCS) standard.

The participants made an observation that some attributes for referenceValue were missing. As a result, changes were made to add attributes for identifying the MIME type, schema and encoding in a referenceValue object. These changes were needed for providing input in a specific format or returning output in a specific format.

There was a suggestion to add ows:additionalProperties and ows:context to metadata. It was observed that ows:additionalParameters allows a service to provide KVP metadata information. It was also observed that Testbed-13 and Testbed-14 had demonstrated the utility of such a capability. As a result, the hackathon participants committed changes to the OGC API - Process specification adding the definition of additionalParameters.

There was also a discussion about whether process output arrays are fully supported. The participants observed that currently the output-value-cardinality requirement limits cardinality of the output to one. A number of workarounds were suggested including returning a list of outputs, returning an archive (zip), returning a Metalink, or returning Multipart/mixed responses. The participants concluded that a solution would need to handle all kind of outputs, including outputs by reference, as well as binary files.

There was a discussion about how to specify support for synchronous execution. The options identified included i) either use a query parameter or a HTTP header to specify the execution mode, ii) return a result object, iii) do not return a header with the location (as there technically is no location). This issue was also related to the broader issue of how to choose between synchronous and asynchronous mode for job creation. The participants observed that indeed the WPS 2.0 specification has a jobID in the the status object, and therefore the OGC API - Processes specification should also include a jobID into the statusInfo object.

One of the observations made by the participants was in relation to Cross-Origin Resource Sharing (CORS) which allows a server to negotiate the locations and types of requests that are available to clients. Connecting to services on other servers requires the cooperation and adherence to the imposed rules by both sides to establish communication and safe operation. Various specific issues and bugs in the respective use of CORS headers and HTTP statuses were identified, analyzed and rectified during the Hackathon, immediately benefiting the server implementers. Some recommendations regarding CORS are presented in the subsection A.25.4 in the appendix.

There was a suggestion for an extension supporting triggers according to job status. In particular the suggestion was for the user/client to be able to specify triggers for conditions like: onSuccess a Url to be triggered upon process completion, or onFail a URL to be triggered on process failure, progressUpdate a URL to be triggered by the job while progressing and should contain progress status. Participants considered whether such a capability might be more appropriate as an extension, perhaps associated with a pub/sub notification capability.

There was a suggestion to add the exception information to the status information at GET /processes/{id}/jobs/{jobID} and remove the GET /processes/{id}/jobs/{jobID}/exception endpoint. The participants considered whether an HTTP 201 code should be returned with POST /processes/{id}/jobs. The participants resolved that allowing for plural form results could address this issue, for example processes/{processId}/jobs/{jobId}/results/{resultId}. Use of an HTTP 201 code was however inconclusive.

There was a suggestion for highly scalable and flexible processes that can be chained together and are centered around 'Collections' as inputs & outputs.

The participants identified three roots for API building blocks, namely the root of the service, collection ID, and collection ID combined the coverage ID. A need to combine these root paths with maps and tiles was identified. There was an observation made that if the roots are combined with maps and tiles, there is an opportunity to provide much more information through data tiles such as vector tiles or coverage tiles. There was also an observation made that if a style and style ID are concatenated with the path, then the API would be able to combine data tiles with portrayals of the information. There was a third alternative identified, which is the concatenation of both the aforementioned approaches. These paths have specific query parameters associated with them. Participants observed that such a capability may not be fully supported by OpenAPI 3.0.

There was a discussion about the Map Tiles API being split up into separate building blocks that can be used independently (e.g., together with either Features and/or Coverages APIs), or in combination with each other and how to divide the specification into modules (requirement classes). The justification for the suggested separation was that tiles can be used to deliver data, whether it is a rendered map, vector features, or a gridded coverage. Similarly, rendered maps can be used independently from tiles. This topic was not resolved during the hackathon.

There was a discussion about map retrieval being in fact a process that takes in one or more collections as inputs, parameters such as styles and producing a Red-Green-Blue-Alpha (RGBA) image "collection" as an output. The collections could be collections of vector features, coverages — collections of gridded cells, or images — collections of RGBA pixels.

There was a discussion about whether the response for maps should return raster or vector maps. Related to this was whether a map was at the same level as a collection. There was agreement that maps are not at the same level as collections.

There was a suggestion to normalize the case used by attributes of the tileMatrix construct. Before the hackathon some of the attribute names were in UpperCamelCase whereas others were in lowerCamelCase. Normalizing the attribute names would ensure consistency of naming and thus make it easier for developers to implement.

There was also an observation made that there is a need to determine where metadata fields supported by WMS could be applied in the OGC API - Map Tiles standard. The metadata fields supported by the WMS standard include: Name, Title, Abstract, KeywordList, Style, CRS, EX_GeographicBoundingBox, BoundingBox, Dimension, Attribution, AuthorityURL, Identifier, MetadataURL, DataURL, FeatureListURL, MinScaleDenominator, MaxScaleDenominator, queryable, cascaded, opaque, noSubsets, fixedWidth, and fixedHeight.

The participants observed that the extents and bounding box classes used by the OGC API - Map Tiles specification have been defined in different ways. The extent class defines a bounding extent as an array of numbers indicating bounding coordinates, whereas the boundingBox class defines the bounding coordinates as separate attributes for the lower and upper corner coordinates.

The participants agreed that OGC API - Coverages should inherit from OGC API - Common as much as possible. This meant that wherever a requirement is specified in OGC API - Common, if it is applicable to coverages, the OGC API - Coverages specification would reference the requirement in the OGC API - Common specification.

The participants agreed that a request for the coverages path would return a list of all of the coverage identifiers included in the collection.

There was also agreement that a request for the coverage description would only return the essential information instead of the complete metadata associated with the coverage.

There was discussion about how to support bounding box (BBOX) filters on multidimensional coverages. The participants expressed the need to inherit the BBOX and time parameters from OGC API - Common, however they also observed that there would be a need to identify a CRS for height. One of the suggestions was for each axis to have a separate coverage filter. The participants concluded that there is currently no construct in the Core part of the OGC API - Common specification that supports filtering of coverages.

There was discussion about the retrieval of coverages. The OGC API - Coverages specification was updated to allow for different ways for getting the coverages individually. Since not every format is suitable for transferring all of the coverage information, participants identified different ways for getting the different types of coverages. It was also noted that for applications that do not want to use collections, they can just use the coverages/{coverageid} path.

There was a discussion about whether parameters, values and URL bases were case sensitive. This issue was observed to be applicable to all of the specifications. There was a suggestion that the OGC API - Common specification should specify a rule for case sensitivity and that that rule should be consistent with the relevant standard of the Internet Engineering Task Force (IETF).

All discussion points that could not be resolved where recorded as GitHub issues like https://github.com/opengeospatial/oapi_common/issues/29 or https://github.com/opengeospatial/oapi_common/issues/36.

There was a discussion about whether OGC API - Common should support the CRS:84 coordinate system (WGS84) by default. The participants observed that the collectionInfo metadata (returned for each collection) allows one to specify the CRS supported by the collection. The client can specify one of the other CRS if they do not support the default. For coverages, the default CRS was observed to be the native CRS. The participants concluded that there will be a default CRS for the API and the OGC API - Common specification should have complete control over the CRS and the default CRS should not constrain the resource.

The participants discussed what role version numbers would play within an OGC API, recognizing that the current suite of OGC web service standards require implementations to indicate the supported versions as a query parameter. The participants determined that the version of the standard would be reflected in the conformance class. Each conformance class would be made uniquely identifiable and any change to that conformance class would result in the creation of a new conformance class. It would therefore not be necessary to indicate the version on the standard on the path or the query parameters of an implementation of the OGC APIs. In addition, it should be the API that includes a version in its path, but not the OGC API paths as these are just sub-resources of the API and many APIs will include other resources not specified by OGC API standards.

The participants observed that there is a need for a CRS that is based on CRS:84 but that includes ellipsoidal height. EPSG:4979 and EPSG:4327 were suggested as a possible basis for such a CRS, but these use latitude/longitude as the axis order. It was agreed to discuss the issue of a height or elevation CRS with the WFS/FES SWG and the CRS Domain Working Group (DWG) at the OGC TC meeting in Leuven. The WFS/FES SWG passed a motion at the OGC TC meeting in Leuven, the week after the hackathon, proposing a new CRS named CRS84h that would be based on CRS:84 and include an additional axis for ellipsoidal height.

At the time of publishing this engineering report, the new CRS named CRS84h has been approved by the OGC Naming Authority (OGC-NA) Subcommittee of the TC and consequently published through the OGC Definitions Server. This achievement shows how coordination between a hackathon, DWG, SWG, the OGC Naming Authority (OGC-NA), and wider TC can lead to rapid results.

There was a discussion on whether the API definition resource should be mandatory at path /api across all OGC API specifications. Earlier in the hackathon it was observed that some of the OpenAPI examples included the /api resource whereas others did not. Participants supporting the addition of /api as a mandatory path in the OpenAPI definitions pointed out that the resource would ensure that the API definition is always a complete representation of the server implementation. In contrast, participants supporting the optional use of the /api resource pointed out that the API definition can already be found through the use of the rel="service" link provided by the landing page and that clients do not need a resource for the OpenAPI definition in the OpenAPI definition as they already have it. It was therefore agreed that there would be no requirement for providing the API definition at path /api or to include it in the API definition itself.

Several Technology Integration Experiments (TIE) were completed during the Hackathon. Table 3 shows the services and client applications that were deployed and bound together during the hackathon. The table also identifies the OGC APIs that were implemented to achieve the integration.

One of the key findings of the hackathon was that the draft OGC API standards are implementable. This was evidenced by the implementations developed by many of the participating organizations at the hackathon.

Another key finding is that the 'building-block' principle, adopted by the different draft standards, allows for a common core to be specified. This was evidenced in the use of common capabilities such as /collection paths and support for bounding box (BBOX) queries.

Another key finding is that when interfaces built on a combination of HTTP, HTML and JSON are standardized across different geospatial applications, the interfaces can be viable enablers of interoperability between the applications. This was evidenced by the successful TIEs achieved during the hackathon.

Additional findings are presented in the following subsections.

The first objective of the hackathon was to develop, deploy and test services/clients that support OGC APIs. The hackathon participants were able to develop and deploy services and clients during the hackathon, as documented in Table 3. The participants were also able to successfully bind together many of the services with client applications provided by other participants, using OGC APIs. This confirms that the first objective of the hackathon was fully met.

A second objective of the hackathon was to suggest improvements for a common core. Some of the discussions around the OGC API - Common specification included default support for the CRS:84 coordinate system and the role of version numbers. The discussion that began shortly before the hackathon on the need for a new CRS that adds ellipsoidal height to CRS:84 was progressed during the hackathon and has resulted in a proposal being passed by the WFS/FES SWG and approved by the TC. The new CRS has been approved by the OGC-NA for publishing on the OGC Definitions Server. Further, the discussion on the role of the /api resource has resulted in consensus between the various SWGs that the API definition does not have to be resource inside the API at path /api. These discussions and the resolutions resulting from them confirm that the second objective of the hackathon was also met.

A third objective of the hackathon was to define rules and guidance that can be documented. Several edits to the OGC API specifications were made on the lead up to the hackathon, and during the event. These edits included additions and deletions of some of the rules and guidance in the specifications, as well as improvements to some of the existing rules and guidance. Github statistics show that more than 4600 additions and 3200 deletions were made to the draft API specifications as an immediate result of the hackathon, thereby confirming that the third objective of the hackathon was also met.

A fourth objective of the hackathon was to validate work that has been completed to date. The successful implementation, by multiple different organizations, of the OGC API specifications supported the validation of the prior work (i.e., the approach taken for the various OGC API specifications). The hackathon also provided the opportunity to invalidate or rethink specific aspects of some of the specifications. The fact that the different specifications extended the OGC API - Common specification, supports the view that the approach taken for organizing and structuring the OGC API specifications was validated by the hackathon. This confirms that the fourth objective of the hackathon was also met.

A fifth objective of the hackathon was to contribute to the GitHub repositories. Figure 9 presents the total number of commits made to Github repositories for OGC APIs for Processes, Map Tiles, Common and Coverages. Figure 8 presents the total number of issues made to Github repositories for OGC APIs for Processes, Map Tiles, Common and Coverages. The commits and issues from the lead up to the hackathon and during the hackathon confirm that the fifth objective of the hackathon was also met.

This subsection has reflected on all of the objectives of the hackathon and confirmed that they were all met. The next section identifies lessons learnt from holding the hackathon.

Based on the results and findings of this hackathon, this engineering report makes the following recommendations.

Work on OGC APIs will continue within the SWGs, during teleconferences and OGC TC meetings. Additional activities are already being planned to support some of the work. Some of the initiatives being planned include the following.

A detailed roadmap for OGC API development will be published by the end of 2019.

52°North is an open international network of research, industry and public administration organizations that partner in a collaborative process of geomatics R&D. All software developed within the 52°North R&D process is published under an open source license.

The Centre for Ecological Research and Forestry Applications (CREAF) is a public research institute created in 1987 and located in Catalonia. It is part of the Autonomous University of Barcelona (UAB). CREAF’s main objective is to generate knowledge and create new methodological tools in the fields of environmental sciences and ecology. CREAF’s research activities are broader than a core on function and diversity of natural ecosystems and also include research and development on big data tools, geospatial standards and data quality.

The CREAF Grumets research team is a multidisciplinary group of people that developed the MiraMon GIS&RS that has been moved to the web using advanced international standards and imaginative innovation strategies.

CREAF is further committed to comply to and improve Open Standards particularly those of the Open Geospatial Consortium (OGC) (http://www.opengeospatial.org/).

The participants from CREAF are Núria Julià, lead developer of MiraMon software, and Joan Masó, co-chair of the WMS and OWS Common SWGs at OGC.

CubeWerx has been involved in the OGC for more than 20 years and has either led or participated significantly in the development or almost every OGC web service specification. CubeWerx is a strong supporter of the current activities in OGC to move from the "old web architecture" (i.e., XML-POST or KVP-GET) to the new OGC API framework initiated by the work done recently by the WFS/FES SWG.

At the moment, CubeWerx are:

Division 7 of the Cologne Government Regional Office is North Rhine-Westphalia’s central service for geographic reference data and top-quality geo-products and services. Division 7 has statewide responsibility for collecting, storing, handling and making available basic topographic geodata. The basic geodata provided by Division 7 (coordinates, elevations, property boundaries, size and use of real estate, topographic information etc.) are needed for a wide range of purposes including planning, construction projects, transport and the supply of basic community services, nature conservation and environmental protection, valuation of real property, real estate transactions and mortgage borrowing. Against a background of rapid technological progress and changing environmental awareness, the division’s services are now in demand from new sections of the community. Rather than chiefly asking for analogue data such as maps, plans and lists, users increasingly prefer digital information which they can integrate into existing databases.

Ecere Corporation has been supporting the OGC API development through participation in various Innovation Program activities (e.g. Testbeds, Vector Tiles and Routing pilots) and working groups. Ecere offers the GNOSIS cross-platform geospatial visualization tools, leveraging GPU hardware acceleration with support for 3D, Augmented, Virtual and Mixed Reality. The technology is built on top of its open-source cross-platform Ecere SDK and eC programming language. As part of its geospatial suite, Ecere offers a map server, a GIS tool as well as a Software Development Kit with support for OGC services (from both the client and server sides).