OGC Geospatial to the Edge Plugfest Engineering Report

The geospatial communities supporting defense, emergency response, and intelligence rely on geospatial data and open standards to accomplish their missions. Profiles of standards and data models were used to ensure sharing of data meet their specific needs. Profiles provide strict implementation guidance to ensure interoperability of geospatial systems and data in these highly specialized and demanding environments. Implementations following profiles compliant with open standards support mission critical operations for enhancing effective and efficient execution.

A Plugfest is an initiative of the OGC Innovation Program. Plugfests provides the right venue for sponsors and technology implementers to come together in a collaborative agile process to solve geospatial challenges. The Plugfest assisted tool enhancement and provided guidance to improve the delivery of enterprise geospatial data to end users. In this initiative, a Plugfest was used to bring more than thirteen data/service producers and clients of data following NSG profiles. The plugfest help discover implementation issues and advance executable test suites.

Before the Plugfest, very few implementations were able to interact with NSG profiles of OGC standards. This is commonly the case when communities want to restrict a rule from the base standard or want to extend what the base standard offers. Support for NSG profiles is not commonly a feature that comes packaged in software products. After the Plugfest more implementations became available that implement the NSG profiles for GeoPackage, WMS, WFS, and WMTS.

The profiles implemented in the Plugfest had corresponding executable test suites. These profiles test suites were in beta by the end of the initiative, ready to be move forward for public release by the OGC Technical Committee. Feedback related to the executable test suites was provided by the participants. In particular, the GeoPackage test was improved during the Plugfest.

Plugfests should be performed for any new profile of an existing OGC standard or any new candidate OGC standard including revisions. Allowing participants to come together to solve interoperability issues has high value in maturing and stress testing implementations of profiles or candidate standards. The result, of high value to the geospatial community, is improvement of the standards and advancement of test suites.

Recommendations for implementing the profiles in regards to the specific profiles are detailed in the Results and Recommendations Section.

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

The following normative documents are referenced in this document.

The goal of this OGC Plugfest was to increase interoperability of information systems using community defined profiles. The Plugfest assisted in tool enhancement and provided guidance to improve the delivery of enterprise geospatial data to end users.

The profiles used in the initiative were:

The participants in the Plugfest took the following roles:

Sprints happened virtually allowing participants around the world to participate and to minimize costs related to travel. Each sprint lasted for 1 week. During the sprint week participants were available to respond to inquiries posted by other participants. In the sprint time clients performed a set of operations following a scenario and documented their success and failures.

Two sprints were executed in this Plugfest.

Sprint 1 tested existing commercial and/or open source products against the scenarios. A limited (not for public dissemination) report on Sprint 1 findings, not revealing vendor information, was shared with the Sponsors and relevant OGC Standards Working Groups. The findings that include the need for improvement in data structure, servers and clients were documented.

Sprint 2 Occurred 3 months after Sprint 1 was completed and documented. Sprint 2 repeated the tests performed in Sprint 1. Participants had three months to improve their software to better create the GeoPackage files and improve their services and clients based on the feedback from Sprint 1.

Two types of geospatial data, based on NSG Profiles, were used in the plugfest:

Each user (client) had to test a data or server and then answer a set of questions. The questions for the vector and raster sources are summarized in this section.

The results of Sprint 1 were summarized in the following table.

The first row provides the label used to identify each client in an anonymized way. For example: Client E, Client L, etc. In total, ten clients were used in the Plugfest.

The first column identifies the sources of data and servers. The contributions were also anonymized. They were sorted in the table by the type of contribution. For example, the Plugfest had 3 WMTS servers: Pluto, Calypso, and Oberon.

The red marks indicate that the client was not able to interact with the server provider. Overall most servers were able to communicate with the clients.

The results for Sprint 2 are shown in the Sprint 2 Results table. Similar to the previous table, it shows if a client was able to interact with the server provider. In addition, it was also captured by shading the cells green if the client was able to successfully perform all the tests with a particular source.

In the following figures are some examples provided by the participants.

Vector queries in the FME Client can be performed two ways.

1 - Interactively using FME Data Inspector as the client alone.

2 - Using the FME Workbench with a workspace script to automate the process.

When FME is used to read a GeoPackage raster tile dataset, the Data Inspector client optimizes the display by balancing the displayed resolution with the zoom level. Unless a specific zoom level is chosen, FME automatically chooses the highest resolution zoom level that can be displayed at the extents chosen, and then resamples as needed.

ArcGIS Desktop was used to Create Mosaic Dataset (Data Management Tools) and to add the images into the dataset. When adding the images the default parameters were kept including the calculation of raster statistics. With the calculation of the statistics, the mosaics remain interactive and available to further analysis.

ArcGIS Desktop was used for publishing the Map Services with allows for the creation of 1-22 zoom levels. The default values were kept for all services.

For the creation of the GeoPackages, Esri turn to the Data Interoperability Tool as opposed to the Add Raster To GeoPackage (Conversion Tool). Work is in progress to make the creation of GeoPackage files more straight forward, in particular, in ArcGIS Pro.

Esri built the queries into the JavaScript and .NET apps, which is easy to use by non-experts. In ArcGIS Pro, the SQL statements were copied and referred back to them for each data source. Nothing special was done to speed up the return of the requests.

Esri stated that setting up the raster and vector GeoPackages, the WMS, WMTS, and WFS was fairly straight forward. Feedback was provided related to test engine irregularities. Esri achieved the goal to reduce the number of errors found in the NSG profiles.

During this experiment two services where provided: WFS and WMTS, both based on the correspondent NSG profiles. The two services were made available with a single GeoServer instance and the necessary GeoServer NSG extensions and plugins, providing a different end-point for each service, i.e. WFS and WMTS.

The provided vector and raster data was also configured in the server. Vector data was stored in a PostgreSQL database in the server. The database schema was adapted to support the NSG versioning needs. Auxiliary world files (.wld) where created for the raster data and directly stored on the file system and served through image mosaic GeoServer extension.

Clients tests feedback and the follow up was done with the support of GitHub issues. The provided WFS and WMTS services where respectively tagged as WFS_NEPTUNE and WMTS_CALYPSO. A total of six issues where reported for the WFS service and three issues for the WMTS service (in both sprints).

The raster data was published using GeoServer image mosaic extension, which allows the user to publish a mosaic from a number of georeferenced rasters. An auxiliary world file (.wld) was created for each granule, and then an image mosaic datastore pointing to the granules directory was created in GeoServer.

The already available image overviews were used as-is, image mosaic takes care of matching the correct overview with the requested zoom level.

Tool ​ogr2ogr w​as used to insert the provided vector data into the PostgreSQL database and gdalinfo was used to get the necessary information to complete the auxiliary world files (.wld) content for each granule.

The raster files were already optimized, e.g. tiled, compressed and with overviews (zoom levels). For vector data, an index was created for each primary key column of each dataset.

When configuring the tile matrix sets for a certain layer, special care should be taken to select only tile matrix sets that make sense for the layer. By default all the tile matrix sets defined by the WMTS NSG profile were available.

In a distributed initiative like this one, the ability to provide the necessary feedback in a concise and straightforward way and encourage discussion happen between all interested parts is fundamental. It is also important to be able to keep track of what happened and be able to get a quick status overview, e.g., show all of the issues related to WFS.

GitHub issues was a good choice for tracking issues. The simple User Interface (UI) makes GitHub issues easy to use by both technical and non-technical people. The labels mechanism provide a good way of managing the issues and the discussion mechanism (with the associated notifications mechanism) is very efficient to use. When creating an issue and assigning it to the interested persons, GitHub will take care of notifying those persons, making them aware of that issue.

In an ideal world, everyone involved with an issue should be able to reproduce that issue in their own environment (debuggable environment), unfortunately this is usually not the case. People work on different environments, they don’t have access to the same clients or servers, etc. This means that special care should be taken when describing an issue.

Improving the information provided in an issue will help all the interested parties better understand the issue and solve the problem. For example, when describing an issue related with an UI, a simple scree capture visually showing the problem is usually easier to interpret than a verbal description. When describing an issue involving a client invoking a server, the actual request sent by the client to the server is a fundamental piece of information. An alternative means to review the issue is to reproduce the issue with a client that is commonly available, such as QGIS.