Testbed-12 Low Bandwidth & Generalization Engineering Report

This ER provides an analysis of the prototype implementations, approaches, test architectures and performance aspects of geospatial data generalization and compression techniques explored in OGC Testbed 12 and findings. OGC Testbed 12 investigated extending WFS with Efficient XML Interchange (EXI) output formats as a method of providing for WFS data delivery in low bandwidth environments. Also it was investigated, how generalization methods can be encapsulated in WPS processes and how WPS profiles for generalization can be specified.

All questions regarding this document should be directed to the editor or the contributors:

See section 10.

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

The DGIWG - Web Feature Service 2.0 Profile document provides recommended implementation profiles for the ISO 19142:2010 Web Feature Service / Open Geospatial Consortium Web Feature Service Interface Standard (WFS) 2.0 – With Corrigendum. The WFS standard provides an interface allowing requests for geospatial features across the web using platform-independent mechanisms and is independent of the underlying data store. One can think of geospatial features as the "source code" behind a map. Whereas the OGC Web Map Service (WMS) interface or online mapping portals return only an image, which end-users cannot edit or spatially analyze, the WFS provides XML-based Geography Markup Language (GML) as the default payload-encoding for transporting geospatial features. In other words, rather than sharing geographic information at the file level using File Transfer Protocol (FTP), for example, the WFS offers direct fine-grained access to geographic information at the feature and feature property level.

The WFS standard specifies discovery operations, query operations, locking operations, transaction operations and operations to manage stored parameterized query expressions. The WFS interface permits users to access and manipulate geospatial feature information from distributed network sources. Technical specifications sometimes have optional features, such that two conforming implementations may not inter-operate completely due to choosing different sets of optional features to support. Even when no formal optional features exist within a standard, there is still a risk that vendors will not implement functionality that is most important to the military community. Also, some standards contain vague or ambiguous wording thus the development and use of profiles can enforce one possible interpretation.

To limit the number of interpretations by implementers and improve interoperability it is possible to define profiles. In standardization, a profile consists of an agreed-upon subset and specific interpretation of a specification. The intention of the DGIWG WFS 2.0 profiles is to minimize such interoperability issues with a specific view to a military context and to mandate a minimum set of service requirements necessary to ensure usability in an operational coalition environment. These profiles are designed to both increase interoperability between WFS servers and to improve the ease of implementation of the WFS standard.

A survey of DGIWG nations was conducted to determine implementation requirements for WFS. These profiles are in response to those survey results. Nations were asked to identify specific requirements for the type of WFS required (Simple, Basic, Transactional, Transactional with Locking, Manage Stored Queries). Based on the results of this survey the profiles define requirements for both a Basic WFS and a Transactional with Locking WFS. The survey also asked respondents to identify requirements for query filters, bindings, bandwidth constraints, output formats and quality of service.

One of the areas for profile assessment was using WFS in low bandwidth environments. This is because results from previous OGC activities and operational deployments indicate that transferring large volumes of geodata from a WFS over a network with poor or very low bandwidth can take a significant amount of time, and network capacity.

To help meet this challenge OGC Testbed 12 developed prototype implementations and conducted Technology Integration Experiments to assess optimizing data transfer under bandwidth-constraint conditions. This document discusses geospatial data size reduction and compression techniques relevant to JSON over GML, zipped XML, EXI etc. Development and testing in Testbed 12 focused on enhancing WFS for EXI compression, with a focus on capabilities that may be considered for possible DGIWG WFS profiling.

The W3C Recommendation Efficient XML Interchange (EXI) Format 1.0 (Second Edition) is a representation for the Extensible Markup Language (XML) Information Set. EXI is intended to optimize performance and the utilization of computational resources. From a practical viewpoint, EXI is designed to reduce the size of XML data exchanged between computers.

EXI uses a grammar-driven approach designed to achieve 'efficient encodings representations'. Consequently, EXI processors are described by the W3C as ‘relatively simple’ and ‘can be implemented on devices with limited capacity.’ An EXI processor is used by application programs to encode their structured data into EXI streams and/or to decode EXI streams to make the data accessible.

EXI is schema ‘informed’, meaning that it can use available schema information to improve compactness and performance. However, the W3C indicates that EXI does not depend on accurate, complete or current schemas to work – a statement which must be considered carefully when using EXI for geographic feature data.

Review of the references above indicates the W3C describes EXI as not 'dependent on schemas'. However, prior investigations have assessed that EXI may compress XML more efficiently if schemas exist describing the format of the expected XML. As background, it is important to understand there are two main ways in which EXI encodes XML documents -

EXI uses a set of built-in grammars to encode XML documents and XML fragments when no schema information is available.

The two modes, schema-less and schema-informed, are important for GML and WFS compression because prior reports noted that coordinates in GML 2 may be defined in a schema as string ('text'). Since schema optimization cannot improve on 'text' this may result in poor compression when there are many coordinates in GML data described by a GML 2 schema. However, the report noted that coordinates in GML 3 may be defined in a schema as float. Schema optimization can improve on float, with the results being good compression with lots of coordinates. The key point being that compression performance may be dependent on the design of the schema as well as the XML data itself.

Generalization in GIS is used to reduce the detail in data. With this reduction in detail, also the amount of data can be reduced, thus improving the usage in low bandwidth environments. [1] lists the following twelve categories of operators for cartographic generalization:

Examples for the twelve operators are shown in the following image:

There are three types of geometry for that generalization techniques will be investigated in this ER: point, line and polygon. Several techniques can be applied to each of the geometry types. We chose to investigate the following operations further:

The implementation of the three operations is described in section 8.

In OGC Testbed 12 EXI participants investigated compression techniques for geospatial data sets delivered by WFS Servers and Clients by augmenting WFS with software capable of producing an output format in EXI (described above).

The testing architecture for WFS Compression in OGC Testbed 12 was configured using a combination of the following components -

These components were configured for testing Compression WFS and EXI as described in the following sequence diagram –