The OGC CDB Volume 1: Core specifies requirements that all GeoPackages that are to be stored in a CDB store shall implement.

The following normative documents contain provisions that, through reference in this text, constitute provisions of this document. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. For undated references, the latest edition of the normative document referred to applies.

ISO / TC 211: ISO 19115-1:2014 Geographic information — Metadata — Part 1: Fundamentals (2014)

OGC: OGC 15-113r5, Volume 1: OGC CDB Core Standard: Model and Physical Data Store Structure (2017)

OGC: OGC 16-070r3, Volume 4: OGC CDB Best Practice use of Shapefiles for Vector Data Storage (2017)

OGC: OGC 12-128r15, OGC GeoPackage Encoding Standard - with Corrigendum (2018)

OGC: OGC ???, OGC Vector GeoPackage Extension (TBD)

This document uses the terms defined in Sub-clause 5.3 of [OGC 06-121r8], which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this Best Practice.

For the purposes of this document, the following additional terms and definitions apply.

OGC CDB and GeoPackage are both standards increasingly used in M&S and GEOINT, but they both contain weaknesses and strengths when it comes to the combined needs of both industries. The combined technology of GeoPackage and OGC CDB is hoped to be a deterministic repository of easily read geospatial datasets suitable for storage, runtime access, and dissemination for live, virtual, constructive, gaming, and Mission Command (MC) systems. The use of CDB raised concerns in both industries because it utilizes Esri Shapefiles and Presagis OpenFlight, both of which are now open standards but were once proprietary. The primary issue is that Shapefiles leverage the dBase IV format to store feature attribution. To address the limitations of ESRI Shapefiles within CDB, potential designs were explored to integrate OGC GeoPackage with OGC CDB. GeoPackage technology was designed to ensure Mobile / Handheld devices could interoperate with systems within the larger Command Post and U.S. Army Enterprise. Current Geopackage adaptations can natively store 3D information, routing information, various types of data ‘tiles’, and other geographically related datasets. An implementation of OGC GeoPackage within CDB aligns CDB vector data storage with the U.S. Army architecture.

Four approaches were testing in the Interoperability Experiment:

Experiment 2 was further broken down into Options 1a – 1d. Each experiment offered its own set of potential advantages, disadvantages, and lessons learned.

Utilizing Option 1c:

Utilizing Option 1d:

The final method selected for conversion of a Shapefile CDB to a GeoPackage CDB is a hybrid of Options 1c and 1d. Because “Off the Shelf” GeoPackage software will not be aware of the class and extended level attributes, we needed to “flatten” the class level attributes with the instance level attributes. It was not possible to effectively flatten the extended level attributes at this time, and those will remain in a separate table with regular table relationships. The decision was made not to use the Related Tables GeoPackage extension.

Several observations were made during the experiments. The output created by the recommended method of conversion allows backward compatibility. However, the GeoPackage CDB results in larger size on disk, which could be problematic. The GeoPackage format (especially as outputted by GDAL) is very space inefficient when it comes to encoding tiles with small quantities of data. But it does reduce the total number of files on disk significantly and eliminates the occurrence of dBase files containing 0 KB.

For more details on the experiments and information above, please see the OGC CDB Vector Data in GeoPackage Interoperability Experiment Engineering Report.

An GeoPackage is an open, standards-based, platform-independent, portable, self-describing, compact format for transferring geospatial information. It is a platform-independent SQLite database file. The GeoPackage specification was developed by OGC. This document describes the encoding of Vector Data in GeoPackage only.

Per Requirement 4 of the CDB Vector GeoPackage Extension, the position of all points is expressed using WGS-84 geographic coordinates (latitude, longitude, altitude), as explained in Volume 8: CDB Spatial Reference Systems Guidance.

As detailed in the following table of GeoPackage Geometry Codes(Table 30 of Appendix G of the OGC GeoPackage Specification), the following geometry types are supported in GeoPackage:

Per Requirement 2 of the CDB Vector GeoPackage Extension, while the GeoPackage model supports encoding of 8 different types that can be stored in the same GeoPackage, the CDB standard requires a maximum of 3 feature geometry types per tile.

The CDB Shapefile standard provides three attribution schemas to represent attribution data. Those three have been flattened into two for use with GeoPackage.

As CNAM in the CDB Shapefile schema was only used as a relationship field, it has been removed from the flattened schema. Figure 1 shows the flattened instance and class-level attributes, and how the extended attributes link to a feature. I_CEAI, I_GEAI, and I_VEAI are all instance-level attributes and are linked to a specific feature. C_CEAI, C_GEAI, and C_VEAI are all class-level attributes and are linked to all features in a layer.

Figure1. CDB GeoPackage Flattened Schema

Example: Record ID 4 in the extended-level attributes table is related to record ID 1 in the linear features table and is an instance-level attribute. Record IDs 5 and 8 are also linked to record ID 4, and all apply to the same linear feature. Note that a 0 in the LNK column denotes the end of the applicable record list. Record 9 in the extended-level attributes table is a class-level attribute related to all features that are present in the highway layer.

All of the information that is needed to instance features is organized in accordance to the CDB tile structure. All the tiled GeoPackage dataset files are located in the same directory. The dataset’s second component selector (CS2) is used to differentiate between files with the same extension or with the same Vector features. Table 1 presents the list of codes that are allocated. Note that Vector datasets do not necessarily use all of the Dataset Component Selector 2 reserved codes. Users of the CDB standard should refer to the appropriate section for an enumeration of the supported File Component Selector 2 codes as well as the ones specific to the Dataset.

The Vector dataset concept and the feature code concepts overlap somewhat; some of the Vector datasets are generalizations or specializations of feature codes. Note that the same feature should not have two representations.

Table 1: Component Selector 2 for Vector Dataset File Names

Table 2: Component Selector 2 for Vector Dataset Table Names

Each GeoPackage file shall contain one and only one feature table. The feature table shall include all attributes (fields) required by the CDB standard (Volume 1, 5.7.1.2. CDB Attribution). The single feature table shall be named identically to the GeoPackage filename (not including the ".gpkg" filename extension). A GeoPackage file may include an optional extended level attribute table. The extended level attribute table shall not contain geometry. The name of the extended level attribute table shall use the CS2 selector shown in Table 2. Table 3 shows the file naming convention.

Table 3: Tiled Dataset File Naming Convention 1

The example below and Table 4 show the filename structure for a road feature. GeoPackage Filename: N13E045_D201_S002_T003_L00_U0_R0.gpkg
Feature Table Name: N13E045_D201_S002_T003_L00_U0_R0
Extended Attribute Table Name: N13E045_D201_S002_T017_L00_U0_R0

Table 4: Basic Filename Structure

Figure 2 shows the flattened schema with filenames as they might appear for a road dataset. The feature and extended attribute tables are shown contained within the GeoPackage file.

Figure2. CDB GeoPackage Flattened Schema Road Dataset Example

See Annex A: Sample Conversion Tools for information on the use of sample conversion tools found in the Cognitics GitHub repo.

Sample conversion tools may be found in the Cognitics GitHub repo located at https://github.com/Cognitics/cdb-shp-geopackage-convert.

Warning: Use the sample conversion code at your own risk. It is intended to provide an example on how to convert Shapefiles to GeoPackage in a CDB, and is not an OGC tested application. Make sure you test on a backup copy of your CDB first.

The sample converter is written in Python, and uses GDAL (version 2.2.3 or greater) with the Python bindings. Binaries of GDAL can be found at https://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries.

To use the sample converter tool, use the following Syntax:

The converter starts by calling the function:

This function scans the CDB to find a list of Shapefiles. It stores a list of all Shapefiles in a file called shapeindex.py. Another file named shapemeta.py is generated with a list of shapefiles and the extents of each file.

The shapemeta.py file is not necessary for the conversion, but can be useful for analysis. That block of code can be removed for a faster conversion if it isn’t needed.

Next, the index file is scanned, and each ShapeFile is processed through:

If the shapefile passed through this function is a feature file, and not a feature class or extended attribute file, the appropriate feature class and extended attributes DBF file (based on Selector2 in the filename) is ready by calling readDBF. All records in each file are read and stored in a dictionary.

Next, GDAL is used to create the appropriate GeoPackage file and layers. Then GDAL is used to open the ShapeFile and iterate through each feature. To flatten the feature class records, as each feature is read, any matching feature class records (based on the CNAM attribute) are added to the feature record.

After copyFeaturesFromShapeToGeoPackage has completed, the extended attribute DBF file that matches the feature file is read. The appropriate layer is created in the GeoPackage. The name of that layer is identical to the filename of the extended attribute DBF file, without the .dbf extension. Each record in the extended attribute DBF file is read and added to the new layer in the GeoPackage. Each record from the DFB file is copied into the GeoPackage layer.

Note that no additional relationships or foreign keys are necessary. The feature attributes that are used to link the features to the Extended Attributes in the GeoPackage are the same as the attributes from the ShapeFile (CEAI,GEAI, and VEAI) and have been renamed to indicate either the Instance or Class level (see Figure 1 in the Flattening the Schema of the Normative document).

After each feature ShapeFile has been processed, the conversion is complete. Note that none of the associated ShapeFile or DBF files are removed from the CDB.