OGC Testbed-13: CDB Engineering Report

This Engineering Report addresses the following requirements:

The work and experiments completed in the OGC Testbed 13 CDB sub-thread are described in this CDB ER document. This ER was reported to the CDB SWG on several occasions and the current key findings of the CDB sub-thread are summarized here:

This ER is mainly of interest to the CDB SWG and the CDB implementation community. In particular, any Change Request Proposals created as a result of the work will be discussed in the SWG. The work documented in this ER is important for CDB SWG as it applies to the CDB 1.0 standard, expands its feature codes/attribution schema using NAS profile, and accesses the CDB data store using OGC web services. Due to the strong link to 3D information streaming and NGA’s NSG application schema, the work is also expected to be of interest to other OGC Domain Working Groups (DWG) such as the 3D Information Management (3DIM) DWG, Defense & Intelligence DWG, Distributed Simulation and Gaming DWG, OGC Web Services and profiling.

All questions regarding this document should be directed to the editor or the contributors:

The following future work items are envisioned:

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

The following figure (Figure 1) describes the CDB workflow executed in Testbed 13.

This document contains the following chapters:

The sample data set used in the CDB sub-thread is the NYC dataset created by FlightSafety International Inc. solely for use by the National Geospatial-Intelligence Agency (NGA) in support of OGC’s Testbed 13 effort. The CDB is physically arranged on disk into the following top level folders:

The Coordinate Reference System (CRS) of this datastore is WGS 84 (World Geodetic System, 1984). The New York data set contains two CDB Geocells: N40-W74 and N40-W75. The demonstration data store has thirteen tiled data layers: Elevation, MinMaxElevation, MaxCulture, Imagery, GeoSpecificFeatures, GeoTypicalFeatures, VectorMaterial, RoadNetwork, RailRoadNetwork, GeoSpecificModelGeometry, GeoSpecificModelTexture, GeoSpecificModelDescriptorbuildings and Hydro. The CDB sub-thread experiments make use of the Elevation, Imagery, Geotypical, and Geospecific model data layers. Each data layer has its own LoDs (level of Details). A CDB-compliant datastore currently uses ShapeFiles for the representation and attribution of vector feature datasets. Aligned with the capabilities of the ShapeFile format, vector features consist of points, lines and polygon features. The raster data layers such as Elevation and Imagery are in GeoTIFF format. The format of the 3D models, the buildings and trees with geometry & textures, is OpenFlight. OpenFlight is a format widely supported in the modeling and simulation community for dynamic and static 3D models. The texture of those models can be described as .rgb format.

In the NYC CDB data version.xml file, it has been denoted that the dataset is compatible with the CDB 3.2 specification. On the other hand, in the Specification_Version.xml file of the NYC CDB datastore, it was mentioned that the CDB Database Version is 3.0 and released in September 2008. After evaluating most of the datasets inside the CDB it was concluded that this sample data store in Testbed 13 was generated based on CDB 3.0 specifications.

Regardless, there are some enhancements in the OGC CDB 1.0 compared CDB 3.2 specification such as updating the terms and definition to be consistent with the ISO 19xxx (Geomatics) series of standards, specifically ISO 19111 Spatial referencing by Coordinates and ISO 19017 Spatial Schema which is published as the OGC CDB 1.0 Standard, Volume 3: OGC CDB Terms and Definitions. Also, the metadata files and folder updated (e.g., corrections to the Feature Data Dictionary, the addition of a Priority field to the Feature Data Dictionary, additional Base Materials for building interiors, changes to the CDB Attributes, the addition of new airliners and countries codes). Handling of Topological Networks, Model LoDs and significant size and some of the datasets had minor changes as well. The list of these updates can be found in the OGC CDB Wiki page [6]. From the CDB 3.0 to the CDB 3.2 specification, updates are more significant and fully described in the OGC CDB Best Practices documents ([7] and [8]).

The underlying use case for the work documented in this report is that:

This section discusses the CDB-related Technology Integrated Experiments (TIEs) conducted in Testbed 13.

Conceptual modelling is a structural methodology for describing how the components of a CDB data store will be implemented based on the modular specification design pattern. The OGC CDB conceptual model presents the important components of the core standard (Figure 2). This model, along with its requirements, extension, file-based structure, data formats, access, and the discovery of services, can be used as the basis for the OGC CDB standard in a variety of application domains. The conceptual model is comprised of concepts, schema, classes and categories, as well as their relationships, which are used to understand, and/or represent an OGC CDB datastore. The following figure (Figure 2) describes the CDB original conceptual model and consists of all the UML files related to the CDB’s structure. The CDB conceptual model could be implemented in Oracle, PostGIS or almost any other database software.

The CDB datastore structure provides efficient access to its contents. The main properties of the CDB datastore UML diagram are listed in the following Table.

The OGC CDB Standard relies on three important means to organize the synthetic environmental data: a) Tiles which organize the data into zones defined by its location with respect to a WGS84 reference system; b) Layers (or datasets) which organize different types of data in a tile; and c) Levels of Detail (LoD) which organize the data in each layer of each tile by its detail. The UML diagram in Figure 3 describes how the data is categorized into tiles, layers and LoDs. This is the basis for the CDB geospatial data categorization. A CDB database can use existing common file formats for storing data such as: TIFF/GeoTIFF (raster data), JPEG 2000 (imagery data), OpenFlight (3D models), Shape (vector data and radar cross sections), RGB (textures), XML (Metadata) and ZIP (file collection).

This diagram shows the general data organization for the CDB. The main properties of the CDB general data organization UML diagram are listed in the following table.

This section describes how the current version of a CDB conformant datastore uses the computer’s native file system to store data in files and directories. An important feature of the CDB Standard is that all CDB file names are unique and that the file name alone is sufficient to infer the path of the file. This is an important performance factor for the simulator client to find the path of a specific feature or dataset by its name. The CDB datastore is composed of several datasets that usually reside in their own directory. However, some datasets share a common structure. The top-level directory of the CDB datastore comprises of the following structures:

Most CDB datasets are organized in a tile structure and stored in the \CDB\Tiles\ directory. The tile structure facilitates access to the information in real-time by any runtime client devices. However, for some datasets that require minimal storage, such as, Moving Models or Geotypical Models, there is no significant advantage to be added for such a tile structure. Such datasets are referred to as global datasets and consist of data elements that are global to the earth.

The OGC CDB 1.0 standard comes with a set of pre-defined feature types, which are listed in a Feature Data Dictionary (FDD), in the form of an XML file and specified as part of the standard. The list of feature types is defined in a Feature_Data_Dictionary.xml [1] file which is located at /CDB/Metadata/Feature_Data_Dictionary.xml. The CDB feature dictionary currently does not support a capability to express relationships between features.

The OGC CDB 1.0 standard uses a convenient categorization of features (based on FACC code which is now called as "CDB feature code"). The first character in a 5-character CDB feature code (e.g., "CCnnn") represents a category of features, the second represents a subcategory of the current category, and the last three characters represent a specific feature type in the subcategory. To provide an even better classification of features, the CDB standard defines an additional attribute called feature sub-code (FSC). By extending the feature code hierarchy structure in this manner, it is possible to define a broader set of feature types. The sub-code value and its meaning depend on the feature code and varies by feature type. The feature code structure and its data model are presented in Figure 4.

The current CDB FDD is a consolidation of the DIGEST, DGIWG, SEDRIS, and UHRB dictionaries. These standards are commonly used for the attribution of source vector data in a broad range of simulation applications (Figure 5). These Feature Codes were supported by the DGIWG FDD and the ISO/IEC 18025 data dictionaries.

Feature codes such as AL015 are an indexing mechanism for CDB compliant applications to access and navigate to Geotypical models (GTModels) more efficiently. Geotypical datasets (directory, geometry, and description) are likely to be referenced multiple times and are stored under a feature code based hierarchy. Here is an example:

The indexing approach greatly simplifies the management of the model library since every model has a pre-established location in the library. The feature codes are also used for naming the files in the CDB folders. In GeoSpecific models, feature codes are used for the file naming in the following data layers: 300_GSModelGeometry and 305_GSModelInteriorGeometry datasets. It looks like:

Moreover, feature codes (FACC and FSC) are two attributes for CDB vector features. They are used for allocation of CDB Attributes to Vector Datasets which is called attribution schema.

To summarize, the only intent of the FDD file is to provide a programmatic way of building a pathname from a FACC code. The other fields (Concept Definition, Recommended Dataset, and Origin) are for information purposes only. These help applications and users understand the meaning of FACC and FSC codes.

Each vector feature is characterized by a set of attributes that are defined in CDB Attributes [2]. For example, a CDB uses Esri Shapefiles to represent vector data and attributes. As per the Esri Shapefile Technical Description, the set of attributes are stored in dBase III+ files for each instance of the vector feature.

Attribution Schemas are the method to handling these different types of attribution. CDB 1.0 supports three attribute schemas to represent attribution data:

Each of the schemas offers different trade-offs in the manner attribution data is accessed and stored. Each of these schemas is largely motivated by storage size considerations and flexibility in the manner attribution data can be assigned to individual features and to groups of features. Attributes are either Mandatory, Optional, not permitted, or not used (Figure 6).

As the following figure (Figure 7) shows, feature codes (FACC and FSC) are two mandatory attributes for CDB vector features.

CDB consumers have two choices when dealing with class-level attributes:

The same reasoning applies to reading the Shapefile containing extended attributes. The user may decide to wait for the presence of values in the CEAI (CDB Extended Attribute Index), GEAI (Geomatics Extended Attribute Index), or VEAI (Vendor Extended Attribute Index) attributes before reading the Shapefile of the extended attributes; or, the user may attempt reading it before parsing the instance or class files in case there are extended attributes referenced by a particular feature. The CDB attribution schema limits the applicability of each of the CDB attributes to specific vector datasets, value ranges and units. This approach helps to reduce the size of the database file of the instance and class-level attribution. This CDB attribute data model is used for the representation of many features using the modeler in real-time simulation.

An application schema includes feature type and model definitions (code, attributes, units, etc.) along with any constraints important for the integrity of the data. Currently, CDB only defines a simple attribute schema for vector features which at the implementation level can be stored in the Shapefile .dbf files. When considering adding support for application schemas in CDB, several aspects need consideration. This is extensively studied in Testbed 13 NAS Profiling ER [3]. These concerns are related to the following topics:

If a CDB data store is to be able to support application schemas, the standard needs to clarify if the feature dictionary and attribute schema are still required and what is their role. The CDB feature data dictionary is currently used in CDB for indexing and accessing the features.

CDB is a file-based data store designed to access and visualize data at different levels of abstraction. It has an internal/physical schema for the indexing of folders and file names for random access. This system is useful for fast access which affects performance, but not semantics. Due to the fixed rules for file naming and file path definition, rapid implementation of new features and changing the indexing structures is difficult to define and implement. However, useful routines can be hardcoded or represented in xml to deal with the issue of physical representation.

Designing a method for supporting additional feature codes and schema should also maintain backwards compatibility as best as possible. Data does nothing in the absence of an interpreter (such as a database generation tool or a client device). As a result, the notion of compatibility should apply not only to CDB itself, but also to the simulation and modeling software that implement the CDB standard. There are actually two types of compatibility that should be considered:

The other important factor with the CDB standard is the performance issue associated with the extended attributes or additional feature data dictionary. Since all the data sources in a CDB data store need to use extended feature attributes, there will be a performance bottleneck in run-time implementations. Therefore, addressing a method for extended feature attributes should address this issue.

The compatibility of the CDB standard with newer versions of feature codes (such as NAS) makes them widely acceptable to major geospatial data/software vendors and stakeholders. Furthermore, this provides a tool for extending the CDB feature codes/attributes and reduces the complexity of discovering, transforming, and streaming geospatial data from the Internet to the simulation environment.

Since the CDB feature codes (based on FACC codes) are ingrained into the structure and naming scheme of the CDB 1.0, the proposed approach in Testbed 13 is based on keeping the current codes as is. In the Testbed 13 CDB sub-thread, another method was applied to convert CDB feature codes and attributes to the NAS profile for accessing and visualizing the New York City CDB sample data. For this purpose, three mapping tables are considered for three feature data sources: tiled vector, Geospecific and Geotypical model dataset. Using these look-up tables, the client device can find the corresponding feature in the NAS profile for each feature instance in CDB vector, Geospecific or Geotypical model datasets. The details of this approach are described in chapter 8 of this ER.

One possible approach for adding NAS attributes to CDB 1.0 is to insert them as the extended-level attributes into the feature attributes. For example, a new attribute can be added to the extended level attribute column of the .dbf files and described by a generated GeomaticAttribute.xml file from NAS profile. This method is carefully investigated in [3].

A practical short-term solution was applied in Testbed 13 to extend the current OGC CDB 1.0 features and attributes (please see the above section). This ER recommends that additional work is required for the replacing of CDB feature codes and attributes. The following future work and changes are expected to be proposed for future CDB versions.

One important change to CDB 1.0 is replacing the feature codes for CDB and updating them with the current technology for complex data modeling, associations and semantics. For example, one solution is to replace the feature data dictionary with Application Schemas (e.g., NAS [3]) that describe the features and attributes in different domains of application with their associations and relationships. To extend the current simple schema of the CDB using NAS (SF0), the NAS feature types are converted into a simple format which is accepted and encoded for the OGC CDB 1.0. Then, these NAS attributes can be added to the feature as an extended attribute.

The critical challenge here is that the indexing technique encoded in CDB (for example the Geotypical model indexing) is based on the FACC codes. Preferably this constraint should be removed. Tis recommendation is based on the discussions in Testbed 13. Another compatible approach may use a mapping to look-up the deprecated FACC codes for each feature type in the CDB.

In Testbed 13, encoding of the application schema can be implemented using the Shapefiles .dbf files for vector features. Moreover, for the 3D visualization performance, one of the difficulties was reading the 3D OpenFlight models. In the NYC CDB datastore, OpenFlight is used for the representation of 3-dimensional geometric representation of all models. Therefore, the following change request is proposed for CDB 1.0 by participants in all of the S3D performance sub-threads.

The above changes may require performing OGC interoperability experiments to better understand the implications of these decisions better. Making these enhancements will allow for the use and implementation of a CDB structured datastore and the extension of its scope to include more 3D simulation and modeling application.

The National System for Geospatial Intelligence (NSG) Application Schema (NAS) specifies a platform independent model for geospatial data and their geospatial semantics specified by the NSG Entity Catalog (NEC) and NSG Feature Data Dictionary (NFDD). The NAS conforms to ISO 19109:2005 Rules for Application Schema, as well as, conceptual schemas specified by other ISO 19100-series standards. NAS includes entity modeling for modeling features, events, names and coverages (e.g., grid, raster, and TIN).

As the NAS specifies an NSG-wide model for geospatial data that supports a wide variety of domains and applications, it is advantageous to define subsets of the NAS that meet specific requirements.

In order to extend CDB using NAS in Testbed 13, an "Urban Military Profile" of the NAS was used as a starting point. This profile defines the vector content requirements for urban-centric profiles. This profile is intended to be used to define the data model requirements for an Urban Military CDB. The S-UTDS subset of the TDS v6.0 Entity Catalog (topographic-profile of an older version of the NAS) has been used as the starting point for the "Urban" Military Profile of the NAS for the purposes of this testbed [3]. NAS has been used as an example of the recent feature data models which includes geospatial data semantics; supports net-centric geospatial services; and is capable of achieving geospatial data interoperability [3].

The 3D model of trees is located in the NYC CDB datastore as a Geotypical model. The Geotypical model is composed of a set of datasets representing its geometry (ModelGeometry and ModelDescriptor) and its textures (ModelTexture, ModelMaterial, and ModelCMT). Similarly, the interior of a model is divided into geometry (ModelInteriorGeometry and ModelInteriorDescriptor) and textures (ModelInteriorTexture, ModelInteriorMaterial, and ModelInteriorCMT) datasets. Here is the path of the Geotypical models of the NYS dataset which are stored in: CDB_NewYork\GTModel\500_GTModelGeometry\E_Vegetation\C_Woodland\030_Trees

In Geotypical model, the indexing approach greatly simplifies the management of the model library since every model has a pre-established location in the library. The following table describes the CDB-NAS feature code mapping for the 3D model of maple trees.

As is presented in table 5, maple trees have different 3D models for different height values. With a closer look into the CDB feature data dictionary file (FDD.xml), the following feature codes are associated with maple trees.

The above table shows that in the OGC CDB 1.0 which inherited the CDB 3.2 specification, the trees are extended to include EC005 which is a point feature instead of the EC030, which is a polygon feature.

The following table shows the feature codes in NSG application schema. There are two fields for defining the features: Definition and Description. The field Definition from the NAS profile is related to the field Concept Definition in CDB. Various types of trees can be defined in the Tree attributes.

Another critical discussion for the CDB to NAS mapping is the attribute schema. When the equivalent features in the CDB and NAS are found, the attributes of these features can be very different. NAS has more information about the attributes, association, geometry and their relationships.

For example, the following table shows the properties of the Tree feature in the NAS profile. This is generated based on the Testbed 13 NAS profile.

For more descriptions about NAS attributes, please see [5].

From the CDB attribution schema (Attributes.xml), it can be seen what are the definitions for attributes and which attributes are deprecated (D), supported (S), not supported (NS) or preferred (P).

For more descriptions about CDB attributes, please see [5].

In the above table, these abbreviations used for determining when an attribute is required for instant/class/extended level: S=Supported, P=Preferred, N S=Not Supported.

For a tree’s Geotypical models, the following tables show some examples of the CDB attributes in class level and instant level.

This section describes the architecture of the CDB (3D) WFS and the design decisions taken to support the envisioned use case. Figure 9 shows the high-level architecture of the WFS. The CDB standard relies on common data formats for its various data types. In case of vector and 3D data, this respectively includes Shapefile and OpenFlight (geometry) and SGI (texture). The contributed CDB (3D) WFS is able to read and serve this data directly from a CDB datastore without any preprocessing. A WFS client should be able to access and visualize the data and align it with the WFS standard without any loss of information residing in the CDB datastore.

Most CDB datasets are organized in a tile structure, following the CDB tiling scheme described in the OGC CDB Core Standard (Volume 1 - document #15-113r3, Section 3.1 and 3.6). Within a tile, data is organized into separate datasets, each including one or more levels of detail. Within OGC Testbed 13, the focus is on offering access to tiled (3D) vector data via the WFS. Although other CDB data types, such as moving models and navigation data were not considered, this is not expected to introduce additional complexity, given the fact that similar data formats were used to represent the data in CDB.

The CDB (3D) WFS was implemented and contributed by Luciad. The implementation builds on Luciad’s COTS software product LuciadFusion, which offers standards-based software components designed for geospatial server development. The following sections discuss the service’s functional and deployment characteristics.

The proposed cloud-based architecture for storing and manipulating CDB datastore is described below. In the following figure, a Java-based CDB API is designed and developed to read CDB datastores from cloud hosting and providers, and generate raster mosaics based on the CDB raster layers. After generation, the raster mosaics are published with WCS using a map-server (e.g. GeoServer). Map server WCS will access and publish each CDB coverage or raster data layer for consumption by WCS compatible clients. The proposed architecture for accessing the CDB using WCS is shown in Figure 11.

WCS provides access to the raster datasets available in a CDB data store. The server can deliver data in the client’s preferred format. The WCS components are described in Figure 12.

This section describes the architecture of the CDB client and the design decisions taken to support the envisioned use case. Figure 15 shows the high-level architecture of the client.

To support the loading of CDB data, the client can connect to two types of data sources:

Additionally, the CDB client offers support to transform local CDB data to the NAS-based CDB Profile.

The CDB client was implemented and contributed by Luciad. The implementation builds on top of Luciad’s COTS software product Luciad Lightspeed, which offers standards-based software components designed for geospatial application development. The following sections discuss the client’s functional and deployment characteristics and include a few screenshots to illustrate the client’s HMI and integration with the New York CDB data source.

The work and experiments that were done in the OGC Testbed-13 CDB sub-thread are described in this CDB ER document. This ER was reported to the CDB SWG on several occasions and the current key findings of the CDB sub-thread are summarized here:

The following recommendations are envisioned for future research to improve upon results of this ER: