OGC Vector Tiles Pilot: GeoPackage 1.2 Vector Tiles Extensions Engineering Report

The Compusult GeoPackage Producer runs as a web-browser based application, as well as being accessible via an OGC Web Processing Service (WPS) instance. The GeoPackage Producer supports producing Mapbox Vector Tile and GeoJSON tile based GeoPackages in a user selected projection system and an uploaded vector source(Shapefile(s),GeoDatabase,SqliteDB etc..). The producer has the ability to convert all feature types into a single Mapbox Vector Tile or produce a single Vector Tile layer for each feature type. Furthermore, the producer also has the ability to embed feature attributes into the Mapbox Vector Tile or to use a GeoPackage Related Tables Extension to produce attributes tables and appropriate mappings for optimal storage. Processed features are converted to Mapbox Vector Tiles (MVT) specification version 2.1 by using the open-source Java library Mapbox Vector Tile Java. Feature types of Polygon and Point geometries have their bounds clipped using a buffer to ensure clients can render freely without having to worry about artificial line segments from tile bounds. Automatic layer order for drawing purposes is detected by examining feature type size, bounds and type. The result of the Vector Tile Extension is a GeoPackage that is compliant to specifications of the National System for Geospatial Intelligence (NSG).

The CubeWerx GeoPackage producer is integrated into the tile-management system for the CubeWerx implementation of the OGC WMTS standard and can generate tiles in different projections and formats from source data of many different formats registered with the CubeSTOR database. The Vector Tiles Pilot added the ability for the underlying tiling system to manage MapBox and GeoJSON vector tiles for the WMTS interface and to write them out into GeoPackages according to the draft Vector Tile Extension Specification. Deflate compression for the vector-tile blobs is also supported and can make the individual tiles a fraction of their uncompressed size.

As recommended by the MVT specification, the CubeWerx-provided MVT tiles clip lines and polygons to the edge of a buffer zone around each tile. By default, we use a buffer zone of 2% of the linear vector-tile size (so a 1024x1024 vector tile would have a buffer zone that’s 20 vector-tile pixels thick), but that percentage is configurable.

Functionality to produce GeoPackages was added to Ecere’s GNOSIS Cartographer GIS desktop application. Multiple data layers can be selected to export into a GeoPackage. The option of tiling vector data using the new vector tiles extensions is provided. A number of options are available, such as whether to embed attributes information within the individual tiles themselves, or store them within an attributes table in the GeoPackage, which is normally done for storing non-tiled vector data. The advantage of the attributes table approach is to avoid repeating the same information for features which occur in many different tiles. This approach also provides the ability to perform SQL queries, potentially together with the R-tree extension for spatial indexing. A choice of tiling schemes is available, including the following:

A selection of supported encodings is provided, including MVT, GeoJSON, GeoECON, GML, and GNOSIS Map Tiles. The user can select whether multiple layers should be encoded as a single MVT set (with multiple layers in each encoded tile), or as a separate tile set per layer. The ability to define and include styles within the GeoPackage, which can be based in part on the attributes associated with the data, was also implemented. Multiple style encodings can be included (GeoPackages produced have also contained Mapbox GL styles, GNOSIS Cascading Map Style Sheets, SLD/SE and GeoCSS).

The Compusult GeoPackage client runs as a web-browser based application, as well as a desktop/android based application. The GeoPackage client supports both Mapbox Vector Tile and GeoJSON tile formats. The client application allows users to select and enable/disable specific WMTS layers as well as overlay any specified base map for visualization. Styles are randomly generated for each layer. The screenshots below show the GeoPackage client rendering different GeoPackage providers. The client also allows for native GetFeatureInfo requests to be sent to a supporting server. GetFeatureInfo is one of the operations offered by Web Map Service (WMS) and WMTS compliant services. Web-browser based applications used a GeoPackage to produce internal WMS/WMTS services that were then rendered by the browser.

GeoPackage.js is an in-browser GeoPackage client published by the National Geospatial-intelligence Agency (NGA). This client is capable of displaying tile tables with image data and feature tables with vector data from a GeoPackage. Image Matters has extended this client to add support for vector tiles encoded using the Mapbox Vector Tile Specification. This extension uses the Leaflet VectorGrid library to render these tiles. This implementation extends VectorGrid.Protobuf from this library to retrieve Protobufs from a GeoPackage rather than from the web. When a user imports a GeoPackage, the user tables are listed in the right pane and may be individually toggled on and off in the displayed map on the left. Expanding the “details” of a table allows a user to view the table metadata, tile bounds, feature attributes, and layers within a vector tile table.

The Vector Tile mobile client creates a tile provider that returns a rendered tile image given a column, row, and zoom level. The client first searches the specified Vector Tiles table in the GeoPackage for a Vector Tile blob at that column, row, and zoom level. The client then parses the BLOB, then looks for styling information. Styling information is included in the GeoPackage as a JSON file. The client then renders the features using that styling information, and saves an image to a tile cache. Layers are saved individually, so that they may be enabled and disabled separately. The map engine can simply display these images at the correct coordinates, which is handled automatically by the tile provider. This version of the client does not have well-specified styling rules. This is something that should be looked at in the future.

Support for GeoPackages and vector tiles extensions as a geospatial data source has been added to Ecere’s GNOSIS software libraries. The capability is available from within Ecere’s GNOSIS Cartographer GIS tool, as well as from any application built using the GNOSIS SDK (whether for desktop, web or mobile). Support for both multiple layers per tiles, or separate tile sets; attributes embedded within the tiles or using attribute tables; for all the different encodings (MVT, GeoJSON, GNOSIS Map Tiles, GML, GeoECON); support for arbitrary tiling schemes; and support for retrieving associated style sheets from within the GeoPackage were implemented.

Vector Tiles

The GeoPackage Vector Tiles extension defines the rules and requirements for encoding vector tiles in a GeoPackage data store.

This extension, like all GeoPackage extensions, is intended to be transparent and to not interfere with GeoPackage-compliant, but non-supporting, software packages.

Image Matters LLC, in collaboration with the participants of the OGC Vector Tiles Pilot.

im_vector_tiles (If this extension is adopted by OGC, then gpkg_vector_tiles will be named as an alias.)

This extension provides new requirements dependent on GeoPackage Clause 2.2 (tiles).

This extension defines an alternate way to encode feature information into a GeoPackage.

read-write

If this extension is in use, then all of the Tiles Option applies. The individual tiles (tile_data in a tile pyramid user data table) are vector tiles.

There are two additional required metadata tables, gpkgext_vt_layers and gpkgext_vt_fields, that mirror the vector_layers key from the JSON object from the metadata from MBTiles. This allows client software to understand the feature schemas without having to open individual tiles.

Mapbox Vector Tiles

The GeoPackage Mapbox Vector Tiles extension defines the rules and requirements for encoding vector tiles in a GeoPackage data store as Mapbox Vector Tiles. This extension is based on the Mapbox Vector Tiles (MVT) specification version 2.1. Note that this format uses Google Protocol Buffers as the content encoding for each tile.

This extension, like all GeoPackage extensions, is intended to be transparent and to not interfere with GeoPackage-compliant, but non-supporting, software packages.

Image Matters LLC, in collaboration with the participants of the OGC Vector Tiles Pilot.

im_vector_tiles_mapbox (If this extension is adopted by OGC, then gpkg_vector_tiles_mapbox will be named as an alias.)

This extension defines an encoding for the Vector Tiles Extension.

This extension defines a specific encoding for Vector Tiles in a GeoPackage.

read-write

If this extension is in use, then all of Vector Tiles Extension applies.

GeoJSON Vector Tiles

The GeoPackage Vector Tiles extension defines the rules and requirements for encoding vector tiles in a GeoPackage data store using The GeoJSON Format.

This extension, like all GeoPackage extensions, is intended to be transparent and to not interfere with GeoPackage-compliant, but non-supporting, software packages.

Image Matters LLC, in collaboration with the participants of the OGC Vector Tiles Pilot.

im_vector_tiles_geojson (If this extension is adopted by OGC, then gpkg_vector_tiles_geojson will be named as an alias.)

This extension defines an encoding for the Vector Tiles Extension.

This extension defines a specific encoding for Vector Tiles in a GeoPackage.

read-write

If this extension is in use, then all of Vector Tiles Extension applies.

OWS Context

This extension provides a mechanism for storing OWS Context content in a GeoPackage. It is aligned with the OWS Context Conceptual Model.

Image Matters LLC, in collaboration with the participants of the OGC Vector Tiles Pilot and the OWS Context SWG.

owscontext (will become gpkg_owscontext if adopted by OGC)

New requirement dependent on Clause 1.

This extension adds an additional level of organization to existing GeoPackage data.

read-write

To use this extension, add the following rows to this table as described in gpkg_extensions.

This table describes extended relationships. The table requires the following columns:

Add a row to this table for each vector tileset that has its own attributes.

A user-defined mapping table describes the many-to-many relationships between base data (tiles) and related data (attributes). A user-defined mapping table requires at least the following columns:

Add a row to this table for each feature/tile instance.

Participants have not taken a close look at the analytical operations that could potentially be supported by vector tiles. Beyond feature visualization and identification, there are many potential pitfalls. Feature information may be modified as you move up or down the tile pyramid for performance reasons. Features may be split across multiple tiles and geometries may be chopped up at tile boundaries. New techniques may be required to fully use this information to its full potential.

This was identified as an area for future work.

Two approaches were considered. One approach created an individual attributes table for each tile. This approach was rejected because it could potentially produce an extreme number of tables. In addition, features can be strewn across multiple tiles and this would make effective querying impossible.

This ER proposes using the Related Tables Extension (RTE) to manage the many-to-many relationship between the attributes and the tiles. This required adding a new requirements class since the draft RTE does not address vector tiles. At this time, it is not known what the performance characteristics of this approach are. This is a possibility for future work.

There is currently no agreed upon way to associate the different layers embedded within a single MVT tile set with their own individual attributes table. One approach that was considered was to add an 'attribute_table_name' column to the gpkgext_vt_layers table. This could be implemented as a change to the Vector Tiles Attributes Extension.

Other formats such as KML have a tight coupling between the data and the style and this causes problems because the data cannot be used for more than one purpose without duplication. The GeoPackage SWG and OWS Context SWG have agreed in principle on a common approach for encoding OWS Contexts into a GeoPackage that will decouple this information. This is documented as OWS Context Extension, which is a minor update to the proposed GeoPackage extension in [2]. This extension provides a way to encode maps (layers, their styles, and associated metadata) into a GeoPackage. This approach is being tested by OGC members for the first time as part of this Pilot.

The second preference is for the style encoding to be tabular in form so that developers can do all of their work in the relational domain and not have to work with document parsing. However, there is not currently a conceptual model that can be implemented in a tabular way. (This is being explored as part of the OGC Testbed-14 project, but at the time of writing, that project was still in progress). For the purposes of this Pilot, the participants have agreed to use existing style encodings such as the Mapbox style encoding. The proposed approach uses the content_type column to define the encoding for the style. This approach is being tested by OGC members for the first time as part of this Pilot.

While OWS Context allows an offering to have multiple styles, the Pilot participants did not foresee a need for this. By eliminating this requirement, an offering can point to the stylesheet it uses and stylesheets can be reused by multiple offerings without a join table.