Publication Date: 2017-05-12

Approval Date: 2017-01-26

Posted Date: 2016-11-16

Reference number of this document: OGC 16-031r1

Reference URL for this document: http://www.opengis.net/doc/PER/t12-A085

Category: Public Engineering Report

Editor: Jeff Yutzler

Title: Testbed-12 GeoPackage Change Request Evaluations

OGC Engineering Report

COPYRIGHT

Copyright © 2017 Open Geospatial Consortium. To obtain additional rights of use, visit http://www.opengeospatial.org/

WARNING

This document is an OGC Public Engineering Report created as a deliverable of an initiative from the OGC Innovation Program (formerly OGC Interoperability Program). It is not an OGC standard and not an official position of the OGC membership.It is distributed for review and comment. It is subject to change without notice and may not be referred to as an OGC Standard. Further, any OGC Engineering Report should not be referenced as required or mandatory technology in procurements. However, the discussions in this document could very well lead to the definition of an OGC Standard.

LICENSE AGREEMENT

Permission is hereby granted by the Open Geospatial Consortium, ("Licensor"), free of charge and subject to the terms set forth below, to any person obtaining a copy of this Intellectual Property and any associated documentation, to deal in the Intellectual Property without restriction (except as set forth below), including without limitation the rights to implement, use, copy, modify, merge, publish, distribute, and/or sublicense copies of the Intellectual Property, and to permit persons to whom the Intellectual Property is furnished to do so, provided that all copyright notices on the intellectual property are retained intact and that each person to whom the Intellectual Property is furnished agrees to the terms of this Agreement.

If you modify the Intellectual Property, all copies of the modified Intellectual Property must include, in addition to the above copyright notice, a notice that the Intellectual Property includes modifications that have not been approved or adopted by LICENSOR.

THIS LICENSE IS A COPYRIGHT LICENSE ONLY, AND DOES NOT CONVEY ANY RIGHTS UNDER ANY PATENTS THAT MAY BE IN FORCE ANYWHERE IN THE WORLD. THE INTELLECTUAL PROPERTY IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE DO NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE INTELLECTUAL PROPERTY WILL MEET YOUR REQUIREMENTS OR THAT THE OPERATION OF THE INTELLECTUAL PROPERTY WILL BE UNINTERRUPTED OR ERROR FREE. ANY USE OF THE INTELLECTUAL PROPERTY SHALL BE MADE ENTIRELY AT THE USER’S OWN RISK. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ANY CONTRIBUTOR OF INTELLECTUAL PROPERTY RIGHTS TO THE INTELLECTUAL PROPERTY BE LIABLE FOR ANY CLAIM, OR ANY DIRECT, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM ANY ALLEGED INFRINGEMENT OR ANY LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR UNDER ANY OTHER LEGAL THEORY, ARISING OUT OF OR IN CONNECTION WITH THE IMPLEMENTATION, USE, COMMERCIALIZATION OR PERFORMANCE OF THIS INTELLECTUAL PROPERTY.

This license is effective until terminated. You may terminate it at any time by destroying the Intellectual Property together with all copies in any form. The license will also terminate if you fail to comply with any term or condition of this Agreement. Except as provided in the following sentence, no such termination of this license shall require the termination of any third party end-user sublicense to the Intellectual Property which is in force as of the date of notice of such termination. In addition, should the Intellectual Property, or the operation of the Intellectual Property, infringe, or in LICENSOR’s sole opinion be likely to infringe, any patent, copyright, trademark or other right of a third party, you agree that LICENSOR, in its sole discretion, may terminate this license without any compensation or liability to you, your licensees or any other party. You agree upon termination of any kind to destroy or cause to be destroyed the Intellectual Property together with all copies in any form, whether held by you or by any third party.

Except as contained in this notice, the name of LICENSOR or of any other holder of a copyright in all or part of the Intellectual Property shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Intellectual Property without prior written authorization of LICENSOR or such copyright holder. LICENSOR is and shall at all times be the sole entity that may authorize you or any third party to use certification marks, trademarks or other special designations to indicate compliance with any LICENSOR standards or specifications.

This Agreement is governed by the laws of the Commonwealth of Massachusetts. The application to this Agreement of the United Nations Convention on Contracts for the International Sale of Goods is hereby expressly excluded. In the event any provision of this Agreement shall be deemed unenforceable, void or invalid, such provision shall be modified so as to make it valid and enforceable, and as so modified the entire Agreement shall remain in full force and effect. No decision, action or inaction by LICENSOR shall be construed to be a waiver of any rights or remedies available to it.

None of the Intellectual Property or underlying information or technology may be downloaded or otherwise exported or reexported in violation of U.S. export laws and regulations. In addition, you are responsible for complying with any local laws in your jurisdiction which may impact your right to import, export or use the Intellectual Property, and you represent that you have complied with any regulations or registration procedures required by applicable law to make this license enforceable.

Abstract

Testbed 12 work has resulted in Change Requests (CRs) to the GeoPackage Encoding Standard. CRs have been submitted to the GeoPackage Standards Working Group (SWG) as GitHub issues. This engineering report (ER) summarizes the results of these activities.

Business Value

This ER shows how the OGC Testbed process furthers the SWG process.

What does this ER mean for the Working Group and OGC in general

OGC Testbeds are an opportunity to evaluate and assess the most problematic parts of the OGC standards baseline. Realistic experiments are often the best way to resolve issues that arise.

How does this ER relate to the work of the Working Group

This ER documents the direct impact that Testbed 12 has had on the SWG work.

Keywords

ogcdocs, testbed-12, GeoPackage, Change Request

Proposed OGC Working Group for Review and Approval

GeoPackage SWG

1. Introduction

1.1. Scope

Several CRs have been submitted against the current GeoPackage 1.1 standard. These change requests require further evaluation to determine the path forward. This ER describes the evaluations of the change requests and the recommendations to the GeoPackage SWG.

1.2. Document contributor contact points

All questions regarding this document should be directed to the editor or the contributors:

Table 1. Contacts
Name Organization

Jeff Yutzler (editor)

Image Matters LLC

1.3. Future Work

No future work is planned to this document.

1.4. Foreword

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

2. References

The following documents are referenced in this document. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. For undated references, the latest edition of the normative document referred to applies.

  • OGC 06-121r9, OGC® Web Services Common Standard

NOTE: This OWS Common Standard contains a list of normative references that are also applicable to this Implementation Standard.

  • OGC 12-128r12, OGC® GeoPackage Encoding Standard

3. Terms and definitions

For the purposes of this report, the definitions specified in Clause 4 of the OWS Common Implementation Standard [OGC 06-121r9] shall apply.

3.1. Abbreviated terms

  • API: Application Program Interface

  • CRS: Coordinate Reference System

  • FK: Foreign Key

  • GPKG: GeoPackage

  • PK: Primary Key

  • SDK: Software Development Kit

  • SQL: Structured Query Language

  • SWG: Standards Working Group

  • WKT: Well-known Text

4. Change Requests

The following table describes the sections that will appear later in this document.

Table 2. Summary of Document Content
Section Number Description Source

4.1

Multi-resolution Vector Data

Testbed-12

4.2

Non-spatial Tables

External

4.3

Deprecating Requirement 69

Testbed-12

4.4

Deprecating non-Interoperable Extensions

Testbed-12

4.5

Column Name for WKT for Coordinate Reference Systems

External

4.6

Add Elevation Extension to standard

GPKG-EE IE

4.7

Remove Minimal Set of Rows from gpkg_spatial_ref_sys

Testbed-12

4.8

Clarification of Acceptable Extensions

External

4.9

Vector Tiling Extension

Testbed-12

4.10

TIN Extension

Testbed-12

4.1. Multi-resolution Vector Data

Reference: GitHub

Vector data is regularly rendered onto a map view. The density of the feature data that can reasonably rendered on a particular map depends on the map’s scale. To support multiple map scales, providers regularly produce multiple versions of the vector geometries.

4.1.1. Status Quo

During previous GeoPackage discussions, a consensus was reached that there should only be a single geometry column in each feature table. The consequence of this decision is that the only way to support multiple geometries for each feature type is to have multiple tables and/or views. However, there is currently no standard way to do this.

4.1.2. Alternatives Considered

  1. Create a set of geometry tables, one for each scale range. (The original design duplicated the attributes as well, but the consensus was that this was wasteful and to use joins to avoid the duplication.) Create a metadata table that links scales to the geometry tables. Client applications then read from the appropriate geometry table.

  2. Create an ancillary geometry table for each feature table. For each feature, create a set of generalized geometries and store in the table the recommended scale denominators for each. Scale ranges may be generated based on visualization requirements. If a generalized geometry differs very little from scale range to scale range, scale ranges can safely be merged to avoid duplication. Client applications then read from the appropriate geometry table.

4.1.3. Recommended Approach

The recommended approach is #2.

There is a benefit to not having fixed scale ranges. For mobile, fixed scale ranges do not necessarily render ideally. This also prevents duplicating generalized geometries that are appropriate for multiple scales.

However, it was determined that it would be premature to encode this into the standard at this time. We recommend researching this topic in Testbed 13 or in a separate interoperability experiment.

4.1.4. Example Table

Table 3. Generalized Geometry Table Data Types
Column Name Type Description Key

id

INTEGER

Primary Key

PK

feature_id

INTEGER

id in Feature user tables

FK

min_scale_denom

REAL

Minimum scale denominator

unique

max_scale_denom

REAL

Maximum scale denominator

unique

geom

BLOB

geometry

4.2. Non-spatial Tables

References: GitHub, Change Request

A strict reading of the GeoPackage specification does not allow non-spatial attribute tables. Data providers need to deliver attribute tables that do not contain geometry properties. For example, a GeoPackage containing real-estate or parcel management data would need to contain:

  • a tiled image of the subject area

  • a feature table of the parcel boundaries (properties such as parcel ID, parcel geometry, area, etc.)

  • a non-spatial attribute table defining property ownership (properties such as parcel ID, purchase date, sale date, owner, etc.)

Based on the existing standard, the non-spatial table cannot be included unless you define the GeoPackage as an "Extended GeoPackage".

Requiring any GeoPackage that contains non-spatial tables to be declared as an "Extended GeoPackage" is not reasonable and does not promote interoperability.

4.2.1. Alternatives Considered

Approach #1: Creating an extension that would make these tables discoverable. A simple extension such as the following would support this:

gpkg_extensions column value

table_name

Actual table name

column_name

NULL

extension_name

gpkg_non_spatial

defintion

TBD

scope

read-write

Approach #2: Redefining the definition of "Extended GeoPackage" and adding a section to the core specification allowing non-spatial attribute tables in a basic GeoPackage would eliminate the problem. This requires some relatively minor updates to sections 1 and 2 and does not introduce any obvious backwards compatibility issues.

4.2.2. Recommendation

We recommend Approach #2.

4.2.3. SWG Response

Approach #2 was adopted by the SWG at the September TC.

4.3. Deprecating Requirement #69

References: GitHub, Change Request

4.3.1. Recommendation

It is difficult, if not impossible, to comply with Requirement 69 "SQL functions that operate on GeoPackageBinary geometries as specified in other extensions SHALL operate correctly on the non-linear geometries specified in this extension." because the functions could have been loaded via an extension such SpatiaLite which cannot be changed, and does not support the non-linear geometries.

4.3.2. SWG Response

The SWG voted to deprecate this requirement. The removal of this requirement will allow for interoperable storage and retrieval of the geometries, while not requiring but allowing existing functions to work with the geometries.

4.4. Deprecating non-Interoperable Extensions

References: GitHub, Change Request

The “User Defined Geometry Types Extension of GeoPackageBinary Geometry Encoding” extension was determined to have the following issues:

  • The geometry encoding is not specified in the extension and therefore a supplemental document explaining the encoding would be required. In the absence of this document, there is no way for an application developer to support this extension and therefore it is not interoperable.

  • Multiple developers could implement the encoding of a new, but similar geometry type such as EllipiticalCurve in different ways.

  • Existing spatial functions will not work with the new geometry types and could potentially cause errors or skip data if used.

4.4.1. SWG Response

The GeoPackage SWG voted to remove this extension for the reasons above.

The SWG also voted to remove two addition extensions, “Geometry Type Triggers” and “Geometry SRS ID Triggers”, from the encoding standard as they directly relate to User-Defined Geometry Types Extension and will no longer be required.

4.4.2. Future Work

The SWG believes that content contained in these extensions would be better suited in a best practice document. This document could outline how to create a complete and interoperable User Defined Geometry Type Extension, including the details of the geometry encoding and how it can be used with existing spatial functions. This would allow two independent developers to create User-Defined Geometry Type extensions that follow the same template and make it easier for clients of the extensions to adopt.

4.5. Column Name for WKT for Coordinate Reference Systems

References: GitHub, Change Request

4.5.1. Recommendation

The “WKT for Coordinate Reference Systems” extension was designed to align to a new OGC Encoding Standard, OGC 12-063r5. The text in GPKG 1.1 (including the column name) incorrectly references “12-163” instead. We recommend updating the reference to the proper value.

4.5.2. SWG Response

The SWG voted to correct the column name. This change corrects all references to the proper “12-063”. The GeoPackage SWG regrets the error. This change is considered to be low-risk. At worst, implementers may need to populate a redundant column to satisfy clients that use this extension but only support GPKG 1.1.

4.6. Remove .gpkx Option

References: GitHub, Change Request

4.6.1. Recommendation

The OGC GeoPackage specification outlines in great detail the extension mechanism. In order for an application to determine what extensions are currently available, it must open the GeoPackage and query the gpkg_extensions table. It is likely that an application that supports no extensions can still access the GeoPackage without issue, especially if the application accesses a GeoPackage in a read-only fashion and the gpkg_extensions.scope column indicates a value of “write_only”. The presence of the ".gpkx" extension provides no real value. Applications still need to open the GeoPackage to determine if the extensions impose any additional requirements on accessing the data and therefore the ".gpkx" extension serves no purpose.

Applications will need to recognize two separate file extensions. There is a distinct possibility that application developers will only support the mandatory ".gpkg" file extension.

4.6.2. SWG Response

The SWG determined that this option served no discernible purpose and therefore voted to eliminate it and strikethrough the text in the standard.

4.7. Remove Minimal Set of Rows from gpkg_spatial_ref_sys

References: GitHub, Change Request

4.7.1. Recommendation

Requirement 11 in the GeoPackage specification provides a minimal listing of rows that shall be contained in the gpkg_spatial_ref_sys table. These rows may not be necessary for the content of the GeoPackage and should not be required. They cause issues for possible profiles of the GeoPackage specification because these profiles may restrict the allowed CRSs, this restriction may not include EPSG:4326. Furthermore, since the allowed CRSs are defined, the value of srs_id of -1 for undefined Cartesian coordinate reference systems and the srs_id of 0 for undefined geographic coordinate reference systems will never be required. It is our recommendation to update this requirement so that the gpkg_spatial_ref_sys table MAY contain these entries.

4.7.2. SWG Response

As of the time of publication, this was still an open issue.

4.8. Clarification of Acceptable Extensions

References: GitHub, Change Request, Change Request.

4.8.1. Recommendation

We recommend that the current GeoPackage 1.1 tables be evaluated and that additional requirements be added stating (as appropriate) that semantics are immutable for specific tables and as such cannot be changed via a profile or extension.

4.8.2. SWG Response

The SWG has had an informal policy position that the definition and semantics of existing columns could not be changed. Therefore it updated Requirement 58 to clarify these rules.

4.9. Add Extension for Vector Tiles

References: GitHub, Change Request

While tiling and the use of multiple levels of details are a proven technique for raster data, it is relatively new for vector data, due to the increased complexity compared to raster tiling and lack of standardization on the topic.

4.9.1. Recommendation

We recommend that a future interoperability experiment or testbed evaluate this extension for interoperability. Once interoperability is proven, it should be considered as an addition to the GeoPackage Encoding Standard as an extension.

4.9.2. SWG Response

As of the time of publication, the SWG had not yet acted on this CR.

4.10. TIN Extension

References: GitHub, Change Request

The current GeoPackage standard is in need of a method for storing elevation data. We recommend a table for triangulated irregular networks (TIN) and an additional table for gridded elevation data.

4.10.1. Recommendation

We recommend that a future interoperability experiment or testbed propose an approach for implementing this capability then evaluating it for interoperability. Once interoperability is proven, it should be considered as an addition to the GeoPackage Encoding Standard as an extension.

4.10.2. SWG Response

As of the time of publication, the SWG had not yet acted on this CR.

Appendix A: Revision History

Table 4. Revision History
Date Release Editor Primary clauses modified Descriptions

June 15, 2016

J. Yutzler

.1

all

initial version

October 20, 2016

J. Yutzler

.2

all

comments integrated

October 28, 2016

J. Yutzler

.3

3, 4

terms updated

November 15, 2016

J. Yutzler

.4

4

added additional CR