Joint OGC OSGeo ASF Code Sprint 2021 Summary Engineering Report

The OGC API - Features standard offers the capability to create, modify, and query spatial data on the Web and specifies requirements and recommendations for APIs that want to follow a standard way of sharing feature data. The specification is a multi-part standard. Part 1, labelled the Core, describes the mandatory capabilities that every implementing service has to support and is restricted to read-access to spatial data that is referenced to the World Geodetic System 1984 (WGS 84) Coordinate Reference System (CRS). Part 2 enables the use of different CRS, in addition to the WGS 84. Additional capabilities that address specific needs will be specified in additional parts. Envisaged future capabilities include, for example, support for creating and modifying data, more complex data models, and richer queries.

The OGC API - Features standard is part of the OGC API family of standards. OGC API standards define modular API building blocks to spatially enable Web APIs in a consistent way. The standards make use of the OpenAPI specification for defining the API building blocks.

This standard specifies the core requirements and extension mechanisms for Discrete Global Grid Systems (DGGS). A DGGS is a spatial reference system that uses a hierarchical tessellation of cells to partition and address the globe. DGGS are characterized by the properties of their cell structure, geo-encoding, quantization strategy and associated mathematical functions. The OGC DGGS Abstract Specification supports the description of standardized DGGS infrastructures that enable the integrated analysis of very large, multi-source, multi-resolution, multi-dimensional, distributed geospatial data. A prototype DGGS API was recently built in the OGC Testbed-16 initiative. It is anticipated that the DGGS API will become a part of the OGC API family of standards.

The GeoAPI Implementation Standard defines the normalized use of the GeoAPI library. The GeoAPI library contains a series of interfaces and classes in the Java programming language defined in several packages which interpret into Java the data model and Unified Modeling Language (UML) types that are specified in ISO and OGC standards documents. The library includes extensive Javadoc code documentation which complements the implementation of the ISO/OGC specifications by explaining particularities of the GeoAPI library: interpretations made of the specifications where there was room for choice, constraints due to the library’s use of Java, or standard patterns of behavior expected by the library, notably in its handling of return types during exceptional situations.

GeoPackage is an open, platform-independent, portable, self-describing, compact format for transferring geospatial information. The GeoPackage Encoding Standard governs the rules and requirements for storing content in an SQLite database. The content may include geospatial and non-geospatial content. The standard defines the schema for a GeoPackage, including table definitions, integrity assertions, format limitations, and content constraints.

The draft OGC API - Common specification documents the set of common practices and shared requirements that have emerged from the development of Resource Oriented Architectures and Web APIs within the OGC. The draft OGC API - Common specification is part of the OGC API family of standards. The specification serves as a common foundation upon which all OGC APIs will be built. Consistent with the architecture of the Web, this specification uses a resource architecture that conforms to principles of Representational State Transfer (REST). The draft OGC API – Common specification establishes a common pattern that leverages the OpenAPI specification for describing APIs.

The OGC API - Coverages specification defines a Web API for accessing coverages that are modeled according to the Coverage Implementation Schema (CIS) 1.1. Coverages are represented by some binary or ASCII serialization, specified by some data (en­coding) format. Arguably the most popular type of coverage is that of a gridded coverage. Gridded coverages have a grid as their domain set describing the direct positions in multi-dimensional coordinate space, depending on the type of grid. Satellite imagery is typically modeled as a gridded coverage, for example. The draft OGC API - Coverages specification is part of the OGC API family of standards.

An Environmental Data Retrieval (EDR) API provides a family of lightweight interfaces to access environmental data resources. Each resource addressed by an EDR API maps to a defined query pattern. The OGC API – Environmental Data Retrieval candidate standard identifies resources, captures compliance classes, and specifies requirements that are applicable to environmental data retrieval. The candidate standard addresses two fundamental operations; discovery and query of environmental data resources. Discovery operations allow the API to be interrogated to determine its capabilities and retrieve information (metadata) about this distribution of a resource. This includes the API definition of the server as well as metadata about the environmental data resources provided by the server. Query operations allow environmental data resources to be retrieved from the underlying data store based upon simple selection criteria, defined by this standard and selected by the client. The OGC API – Environmental Data Retrieval candidate standard is part of the OGC API family of standards. The OGC API – Environmental Data Retrieval candidate standard is part of the OGC API family of standards.

The draft OGC API - Maps standard describes an API that presents maps portraying data that has been rendered according to a style. The maps served by implementations of the draft OGC API - Maps standard are retrieved as images of any size, generated on-the-fly, and with the styling determined by the client application. The draft standard can be considered the successor to the widely implemented WMS standard. The draft OGC API – Maps specification is part of the OGC API family of standards.

The draft OGC API - Processes standard enables the execution of computing processes and the retrieval of metadata describing their purpose and functionality. Typically, these processes combine raster, vector, and/or coverage data with well-defined algorithms to produce new raster, vector, and/or coverage information. The draft OGC API – Processes specification is part of the OGC API family of standards.

OGC API - Records provides discovery and access to metadata records about resources such as features, coverages, tiles / maps, models, assets, services or widgets. The draft specification enables the discovery of geospatial resources by standardizing the way collections of descriptive information about the resources (metadata) are exposed. The draft specification also enables the discovery and sharing of related resources that may be referenced from geospatial resources or their metadata by standardizing the way all kinds of records are exposed and managed. The draft OGC API – Records specification is part of the OGC API family of standards.

OGC API - Routes describes the requirements for interoperable web-based route computation and specifies a number of alternative approaches to fulfill these requirements. One of the approaches is based on the current draft of the draft OGC API - Processes - Part 1: Core specification while the other comprises a specialized API although also based on the draft OGC API - Common - Part 1: Core specification. Both approaches facilitate a common Route Exchange Model (REM) that is based on GeoJSON.

OGC API - Styles describes the interface and exchange of styling parameters and instructions. The construction of symbology components of styles is addressed in the OGC Symbology Conceptual Model: Core Part standard and multiple OGC and other style encoding standards. The draft OGC API – Styles specification is part of the OGC API family of standards.

OGC API - Tiles references the OGC Two Dimensional Tile Matrix Set (TMS) standard. The TMS standard defines the rules and requirements for a tile matrix set as a way to index space based on a set of regular grids defining a domain (tile matrix) for a limited list of scales in a CRS. The draft OGC API – Tiles specification is part of the OGC API family of standards.

GeoNetwork is a catalog application for managing spatially referenced resources. It provides metadata editing and search functions as well as an interactive web map viewer.

GeoNode is a web-based application and platform for developing Geographic Information Systems (GIS) and for deploying Spatial Data Infrastructures (SDI).

GeoServer is a Java-based software server that allows users to view and edit geospatial data. Using open standards by the OGC, GeoServer allows for great flexibility in map creation and data sharing.

The Geographic Resources Analysis Support System (GRASS) is an open source GIS providing raster, vector and geospatial processing capabilities. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R or in the cloud [2].

Mapbender is a web-based geoportal framework to publish, register, view, navigate, monitor and grant secure access to spatial data infrastructure services.

MapServer is an open source platform for publishing spatial data and interactive mapping applications to the web.

OSGeoLive is a self-contained bootable DVD, USB thumb drive or Virtual Machine based on the Lubuntu operating system, that allows users to try a wide variety of open source geospatial software without installing anything.

PostGIS provides spatial objects for the PostgreSQL database, allowing storage and query of information about location and mapping.

PROJ is a generic coordinate transformation software library that transforms geospatial coordinates from one coordinate reference system (CRS) to another.

pycsw is a server-side python implementation of the OGC Catalogue Services for the Web (CSW) standard.

QGIS is a free and open-source cross-platform desktop GIS that supports viewing, editing, and analysis of geospatial data.

GeoHealthCheck is a Python application to support monitoring OGC Web Services uptime and availability.

Map Markup Language (MapML) is a text-based format which allows map authors to encode map information as hypertext documents exchanged over the Uniform Interface of the Web.

GeoTrellis is an open source, geographic data processing library implemented in Scala and designed to work with large geospatial raster data sets.

MapProxy is an open source proxy for geospatial data. It caches, accelerates and transforms data from existing map services and serves different desktop or web GIS client applications.

MobilityDB is a database management system for moving object geospatial trajectories, such as GPS traces.

OWSLib is a Python package for client programming with OGC Web Service (OWS) interface standards, and their related content models.

PDAL is a C++ BSD library for translating and manipulating point cloud data.

pgRouting extends the PostGIS / PostgreSQL geospatial database to provide geospatial routing functionality.

pygeoapi is a Python server implementation of the OGC API suite of standards.

Stetl, Streaming ETL, is an open source (GNU GPL) toolkit for the extraction, transformation and loading (ETL) of geospatial data. Stetl is based on existing ETL tools like GDAL/OGR and XSLT.

ZOO-Project is a Web Processing Service (WPS) implementation written in C. It is an open source platform which implements the WPS 1.0.0 and WPS 2.0.0 standards edited by the Open Geospatial Consortium (OGC).

Apache ActiveMQ™ is an open source, multi-protocol, Java-based messaging server. It supports industry standard protocols so users get the benefits of client choices across a broad range of languages and platforms.

Apache CXF™ is an open source services framework. CXF helps you build and develop services using frontend programming APIs, like JAX-WS and JAX-RS.

A free and open source Java framework for building Semantic Web and Linked Data applications.

Apache Karaf is a small OSGi based runtime which provides a lightweight container onto which various components and applications can be deployed.

Apache Spatial Information System (SIS) is a free software, Java language library for developing geospatial applications. The library is an implementation of OGC GeoAPI 3.0.1 interfaces and can be used for desktop or server applications.

Apache Superset is an open-source software cloud-native application for data exploration and data visualization able to handle data at petabyte scale.

Apache Tomcat is an open-source implementation of the Java Servlet, JavaServer Pages, Java Expression Language and WebSocket technologies.

ldproxy is an implementation of the OGC API family of specifications, inspired on the W3C/OGC Spatial Data on the Web Best Practices. ldproxy is developed by interactive instruments GmbH, written in Java (Source Code) and is typically deployed using docker (DockerHub). The software originally started in 2015 as a Web API for feature data based on WFS 2.0 capabilities. In addition to the JSON/XML encodings, an emphasis is placed on an intuitive HTML representation.

The current version supports WFS 2.0 instances as well as PostgreSQL/PostGIS databases as backends. It implements all conformance classes and recommendations of "OGC API - Features - Part 1: Core" and "OGC API - Features- Part 2: Coordinate Reference Systems By Reference", as well as other draft extensions (including Part 3 and Part 4). ldproxy also has draft implementations for additional resource types (Tiles, Styles).

The CubeWerx server ("cubeserv") is implemented in C and currently implements the following OGC specifications:

The cubeserv executable supports a wide variety of back ends including Oracle, MariaDB, SHAPE files, etc. It also supports a wide array of service-dependent output formats (e.g. GML, GeoJSON, Mapbox Vector Tiles, MapMP, etc.) and coordinate reference systems.

The MiraMon Map Server is a CGI application encoded in C language that is part of the MiraMon Geographic Information System (GIS) & Remote Sensing (RS) suite. The software originally started 10 years ago as a WMS server in support of the Catalan Administration and CREAF data services. Currently the server implements WMS, WMTS and partially implements WFS and WCS. It also partially implements the OGC Sensor Observation Service (SOS) standard. It also includes prototype support for the draft OGC API - Maps and OGC API - Tiles specifications. In order to perform efficiently, it requires a process preparing the data to be offered. The server can interoperate with other vendors' clients. When combined with the MiraMon Map Client, the server offers additional functionality, including functionality recently developed for the Catalan Data Cube. The MiraMon Map Client is built using client-side JavaScript and can therefore run on any web browser.

The GNOSIS Map Server is written in the eC programming language and supports multiple OGC API specifications. GNOSIS Map Server supports multiple encodings including GNOSIS Map Tiles (which can contain either vector data, gridded coverages, imagery, point clouds or 3D meshes), Mapbox Vector Tiles, GeoJSON, GeoECON, GML and MapML. An experimental server is available online at https://maps.ecere.com/ogcapi and has been used in multiple OGC Innovation Program initiatives.

The National Weather Service deployed an instance of the OGC API - Environmental Data Retrieval candidate standard. The implementation offers the following types of data products: MEteorological Terminal Aviation Routine Weather Report (METAR), Terminal Aerodrome Forecast (TAF), Global Forecast System(GFS), North American Mesoscale model (NAM), National Digital Forecast Database (NDFD), Canadian Global Deterministic Prediction System (GDPS), French Action de Recherche Petite Echelle Grande Echelle (ARPEGE), and NASA monthly precipitation and hourly data (originally in HDF5 format).

The implementation is able to ingest GRIB and HDF5 data as well as text products and data from Thredds. The implementation also contains schema support for the following output formats: CoverageJSON, GRIB, and IWXXM, with additional support added during this sprint to add support for NetCDF. In the future, work will be done to support HDF5 as an output format, but NetCDF was decided on to implement first.

Finally, the implementation supports the sampling geometry types of Position, Area, and Cube with additional support added during this sprint for Trajectory.

Amongst the cross-cutting issues discussed during the code sprint was the issue of discovery of landing pages. Currently OGC APIs offer a landing page concept which serves as the entry-point for discovery of other resources offered by an API. The sprint participants highlighted the potential benefit of having a "landing page of landing pages" that could enable a client application to discover other landing pages. Participants also observed that an OGC API - Records implementation could provide the functionality needed to discover landing pages of other OGC APIs.

The participants observed the discovery of landing pages to be particularly useful in a microservice-based architecture. The microservices architecture deploys application code into small and manageable containers that can run independently of others. Whereas the microservices approach offers some benefits such as isolation of many types of faults, it can also introduce challenges in memory consumption as each microservice runs in an independent container.

The sprint participants also discussed the ability of code generation tools to generate code that adequately describes the APIs defined using OpenAPI. Several participants commented that code generation tools seldom describe the business logic required to handle geospatial operations. Therefore, developers often have to manually write code into the stubs created by code generation tools. Further, the participants noted that for APIs that enable the publishing of dynamic resources at runtime, code generation tools often require additional manually written code to enable support of those dynamic resources. In this context, dynamic resources include feature collections that might change their schema at some point in time. The participants therefore recommended that implementers of code generating tools should make enhancements to address some of those shortcomings, for example support for dynamic resources and enabling semantic awareness to provide context to API components.

There was discussion on how to represent the language for a record (possibly using an HTTP resource; the language that was requested). There was discussion on what happens if a client requests all the languages available at the same time. There is the potential to end up with verbose arrays. The participants suggested that a better approach could be to leverage hypermedia.

CubeWerx updated their server during the code sprint to follow the draft OGC API - Records specification. The team made progress on their implementation of support for OpenSearch. There was also discussion on potential integration with ActiveMQ to support implementation of a GeoSynchronization Service (GSS).

The pygeoapi team advanced their implementation of OGC API - Records. There were some updates to the Records API schema. The team also implemented the itemType property to enable identification of the types of items returned by an API. The pygeoapi team also discussed with the GeoNode team about the possibility of using pygeoapi as a backend to GeoNode.

The pycsw team discussed how they are going to implement OGC API - Records, while still supporting versions 2 and 3 of the OGC Catalogue Services for the Web (CSW) standard. CSW is the predecessor of OGC API - Records. CSW supports the ability to publish and search collections of descriptive information (metadata) for data, services, and related information objects.

Participants from interactive instruments updated ldproxy during the code sprint to support the Core, HTML, JSON and CQL Filter conformance classes of the current draft of OGC API - Records (pull request). A sample instance was set up during the sprint and added as an implementation in the GitHub repository of OGC API - Records.

Participants experimented with the implementation of OGC API - Processes through use of the Spring Framework. The Spring Framework is an application framework for the Java platform. Spring has been particularly popular amongst Java developers for enabling the implementation of RESTful APIs and microservices. Process description and process execution were successfully implemented during the code sprint. Some of the additional considerations during the code sprint included workflows, transactions and security.

There was a discussion about supporting the library of UCAR Unidata. The participants sought to make UCAR’s library an optional item that could be used alongside implementations of the GeoAPI. The UCAR library supports netCDF - a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. A GitHub repository was created to support this initiative.

There was also discussion about the possibility of advancing the Python port of the GeoAPI library. Work has already started on the Python port. Other languages were also considered, for example NodeJS. Volunteers were invited to lead GeoAPI efforts on other programming languages.

The MapML team discussed with the GeoServer team about the progress for upgrading from a community module to core. There was some work done on feature queries and navigation. The participants encountered some issues relating to Java 8 and Maven, however the issues were successfully resolved. The team planned to continue server improvement.

The sprint participants discussed how to transition from one approach of serving tiles to another way. The participants noted that the challenge is how to combine a collection with a style. One suggestion was that having the map before the style on the URL may improve access. The GeoServer team expressed their preference for that sequence (i.e. map/style/).

There was discussion about enabling pygeoapi to support xarray databases. An xarray database supports the labelling of data to indicate dimensions, coordinates and other attributes.

On the OWSLib side, there was discussion about doing an OGC API client. One of the options considered for adding to OWSLib was pydantic, a python-based framework for data validation. There was also discussion about doing OWS documentation in jupyter using Sphinx.

There was discussion about the potential to implement an OGC API - Routes interface on top of pgRouting. The participants acknowledged that the scope of pgRouting does not currently include APIs. However, the participants also acknowledged the potential benefit of ensuring that information provided by an OGC API - Routes implementation could be fully ingested by pgRouting and vice versa. Another OGC standard that was identified as potentially relevant to pgRouting is the OGC IndoorGML standard.

Accomplishments during the code sprint:

There was an announcement that an experimental version of a GDAL driver that supports OGC API - Tiles and OGC API - Coverages has recently been implemented. The GDAL driver has been developed to demonstrate work related to the “Modular OGC API Workflows” initiative that is led by Ecere and supported by Natural Resources Canada. Much of the discussion regarding the GDAL driver focused on improving the documentation.

The GeoNetwork team worked on a refactor of the code, related to OGC API - Records. In previous iterations the YAML files provided by OGC were used to build a Java skeleton. With the refactor, the openapi document is generated from Java code using the SpringFox library. This allows the developer to add some properties and methods which are not (yet) in the standard.

For example GeoNetwork provides DCAT, schema.org, Atom outputs of Metadata records in various encodings, such as HTML, JSON, (ISO19139) XML, JSON-LD, RDF/XML and Turtle. The buttons for accessing these additional encodings are shown on the right-hand side of Figure 10.

A list of discussions from the sprint is presented below:

The GeoServer team worked on a number of activities during the sprint. A series of screenshots of GeoServer, from the code sprint, are shown in Figure 18.

Andrea (GeoSolutions) worked on an initial experiment with OGC API - Maps and shared these observations:

GeoServer maintains a community area for experiments (such as prototype implementations of OGC API Maps and OGC API Features). Michel and Jody (GeoCat) looked at how much work would be required on an implementation of OGC API - Features to meet requirements for configuration, documentation and quality assurance.

Improvements were made to the appearance of the HTML pages as shown in Figure 19. The templates used can now be customized on a feature type or workspace basis.

Other improvements included:

Staff from the Canada Centre for Mapping and Earth Observation, a part of Natural Resources Canada (NRCan), worked on MapML using a Leaflet client. A screenshot is shown in Figure 20. This work has been submitted for inclusion in GeoServer.

The participants created the alpha version of GRASS GIS import modules for the OGC API - Features Standard and the draft OGC API - Coverages specification.

QGIS already had OGC API - Features support. Therefore, the QGIS team worked on an OGC API - Records client. A screenshot of the client is shown in Figure 21. A pull request has been prepared at https://github.com/qgis/QGIS/pull/41713

Relevant to this development is the discussion at https://github.com/opengeospatial/ogcapi-records/issues/95. QGIS facilitates addition of the service layer to a map, when it detects the type of the service. The current CSW implementation has a long algorithm that tests various metadata properties, the service URL and probes the endpoint.

The ASF group was interested in using QGIS as a tool for data modification and loading into Fuseki. They were looking for tooling that would export the data as RDF from QGIS. They found a solution using GeoServer as middleware, which has a community plugin at https://docs.geoserver.org/latest/en/user/community/json-ld/output.html, which offers JSON-LD capabilities.

The participants observed that ActiveMQ is relevant to the concepts discussed in the OGC Publish/Subscribe (PubSub) 1.0 standard and the draft PubSub Whitepaper. OGC PubSub 1.0 is an interface specification that supports the core components and concepts of the Publish/Subscribe message exchange pattern with OGC Web Services. Another observation made was that ActiveMQ is potentially related to the draft GeoSynchronization Service (GSS) specification. The draft GSS specification describes a service that allows data collectors to propose changes to be made to a data provider’s features.

The Apache Jena team discussed an approach with the GeoServer team for enabling the export of a feature collection from QGIS to GeoServer, and then to Apache Jena Fuseki. Fuseki offers a SPARQL Server interface on top of Apache Jena. The groups concluded that it would be possible to create a plug-in for QGIS that allows it to post GeoJSON to GeoServer. An additional plug-in could then be implemented to export GeoJSON-LD from GeoServer to Fuseki.

The GeoJSON-LD document would be interpreted as RDF triples by Fuseki. There was also discussion about how to represent the schema of the feature collection. SHACL was identified as a possible candidate for representing the schema of the feature collection. Jena supports SHACL.

As part of ongoing discussions regarding renewing the OSGeo/OGC Memorandum of Understanding (MOU), there was discussion between Scott Simmons (OGC Chief Standards Officer) and members of the OSGeo Board of Directors on assessing the current MOU as well as current state of affairs. In addition, there was discussion on opportunities to strengthen the MOU given the OGC’s increasing focus on developers and the importance of free and open source software.

The draft MOU continues to be updated and will be tabled for review by both organizations.