Testbed-12 LiDAR Streaming Engineering Report

This OGC® document gives guidelines how the OGC Sensor Observation Service 2.0 interface can be used for enabling a JPIP-like streaming of point cloud data. It summarizes the findings gathered during the implementation of a LiDAR Streaming SOS server during the OGC Testbed-12.

All questions regarding this document should be directed to the editor or the contributors:

Based on the findings during the development of the LiDAR streaming SOS, there are several work items we recommend for the future. These comprise:

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

The OGC SWE standards comprise several specifications to allow the integration of sensors and their measurements into spatial data infrastructures. This comprises both, encodings for metadata and observation data as well as Web service interfaces for functionality such as downloading observation data, event subscriptions as well as controlling/tasking sensors. Within the context of streaming access to LiDAR data, three standards are relevant, which are introduced in the following sub-sections:

LAS is a public, binary file format for the interchange of 3D point cloud data. It is one of the most widely used approaches for encoding and transferring point cloud data. LAS is available as an open specification which is published, maintained and copyrighted by the American Society of Photogrammetry and Remote Sensing (ASPRS) [3]. In addition, a compressed version of LAS is available as the LAZ format.

The development described in this ER addresses two main objectives:

The scenario for the demonstration of the LiDAR SOS deals with an offroad routing application that is used in an emergency situation:

Due to an earthquake, it has been necessary to evacuate a part of San Francisco. However, due to the earthquake the road surface has been damaged. There are no alternative roads available and there are already evacuees on the route. How do we get the stranded evacuees to safety? Since there are no roads available, we need a cross-country evacuation route.

For testing and demonstrating the LiDAR SOS implementation, an area in California north of Richmond and Oakland, and South of Vallejo California was selected. This area lies on the most direct route from Oakland to Travis Air Force Base.

The LiDAR data sets were obtained from the USGS as LAS files:

The input data used for the LiDAR SOS developments were provided in the LAS/LAZ format (see previous section).

Connecting to a SOS server for getting service metadata and a description of its data contents is done by calling the GetCapabilities operation. This operation allows clients to retrieve the metadata of the SOS and detailed information of the point clouds that are accessible over the SOS interface.

The contents section describes the service as a collection of offerings. Each offering contains the following essential information:

The content of the extension attribute holds the necessary information of the converting process of a specific point cloud scene to the octree data format. This includes information such as the cubic bounding box, point attributes and the number of hierarchy levels. All this information (especially the cubic bbox) is essential to set up the octree structure at the client.

If a user wants to retrieve the description of the sensor or process that has generated the LiDAR scene, the DescribeSensor operation of the SOS can be used. Within the response of this operation, additional (usually SensorML encoded) metadata from the SIPC/HDF5 file may be returned. Please note: in this testbed a stronger focus was on the development of the streaming so that a comprehensive metadata mapping to SensorML would be part of future work.

The GetFeatureOfInterest operation can be called to retrieve all the features of interest for which the SOS offers data. It accepts a procedure as parameter and returns all the Features of Interests, i.e. nodes within the octree, related to that procedure. This operation provides a sort of list consisting of the featureIDs of all nodes within the octree. All featureIDs always have the same structure like follows: <name>-<octree_identifier>, where the name is any arbitrary name shared for all features and the octree_identifier corresponds to the location within the octree.

In addition, to determine the nodes of the octree that cover a certain area, the GetFeatureOfInterest operation also supports spatial filters within the query.

The GetResultTemplate operation may be used for determining the encoding and the structure of the data in GetResultResponses. However, for this implementation in the context of Testbed-12, the GetResultTemplate operation is not needed.

The GetResult operation is the final operation to request the data of a specific node in the octree. As input parameters it requires the unique identifier (i.e. featureID) of the specific node in octree, which can be requested by the GetFeatureOfInterest operation. In addition, it requires the unique identifier for the observedProperty that can be found in the corresponding offering of the GetCapabilities response.

The GetResult operation returns the requested node of the octree as Base64 encoded LAZ string. Therefore, in order to make use of the encoded LAZ files, it is first required to decode the Base64 string in the resultValues xml attribute.

We use the feature_id to select the subset of point cloud data that shall be returned (e.g., r.01, r.02). This allows clients to dynamically navigate through the structure of an octree to request more detailed data for certain areas if a users zooms in.

Currently the payload of the GetResult response is in the LAZ format. However, the support of further output formats (e.g. WKB, EXI) could be easily enabled by adding a dedicated result format request parameter to the GetResult implementation.

An essential requirement to the streaming-based provision of point cloud data is to enable multi-resolution access to the data. However, since a point cloud scene may contain several million data points, an on-demand level of detail selection and data fetching process may become very CPU intensive and time-consuming.

A predominant approach to tackle these limitations in dealing with large point clouds is to pre-process the input model in order to provide structural access to a hierarchy of pre-computed point clouds with different coverages and resolution-levels. The SOS server makes use of this idea and therefore relies on the octree data format of Potree.

The generalization of the Potree data structure works as follows:

In this way, each point of the initial point cloud is allocated to a different level of details where low-level nodes contain low resolution models while the resolution increases and the covered area decreases at higher levels. This allows the combination of several point cloud levels with a stateless top-down octree traversal to locally improve sample densities.

The Potree file format is actually a collection of nodes and sub-nodes of distinct point clouds, where each node can be addressed by an unique identifier that indicates its exact position in the hierarchy. This identifier looks like the following: Depending on the spatial position of the child nodes relative to its parent node, each child node can be addressed by a number ranging from 0 to 7.

To demonstrate this principle, consider a root node of the octree indexed with the character r. Each of r’s child nodes can now be assigned a fix number ranging from 0 to 7. In this way, the third child of r is addressed by r2 and the first child of r2 is addressed by r20.

With the aforementioned octree-based data format, we can provide a JPIP-like streaming approach as required by the project. One of the major characteristics of JPIP-based streaming is that the server does not send data that is already in the client cache. Instead, only the missing information is sent to the client. Therefore, the server is responsible to identify parts of requests that have already been send to the client in previous responses. This is called cache modelling and requires the server to be stateful. However, the SOS server does not provide any session-based capabilities.

For this reason, we introduced along with the Potree data format a static hierarchy of point clouds that are completely distinct from each other, meaning that each point only exists once in the whole data structure. This way of organizing the data has the benefit that it enables the easy caching of contents transferred by the server in responses without any retransmissions of overlapping requests.

In addition, our approach enables the multi-resolution access to the data similar to JPIP. Clients initially get a low-resolution view of the point cloud and the resolution can be increased by requesting more sub-nodes of the specific part of the scene. By following this approach, the client provides access to a high-resolution image, but only transfers the subset of the point cloud that has a specific resolution and that is actually required by the client. The same intention is followed by JPIP.

LiDAR scenes may consist of hundreds of millions points, which leads to the difficult problem of managing massive number of points in PostgreSQL, because considering each point as a single row in relational databases leads to significant scalability issues. For this reason, we used the PostgreSQL extension pgPointCloud to overcome the limitations of a straight-forward point cloud integration based on single points. It introduces a new datatype called pc_patch that creates an abstraction layer by storing group of several hundred to thousands of points rather than single point entities. By making use of this new datatype, the overhead due to the great number of rows is omitted.

As presented before, we used an octree-based data format for organizing the point clouds and providing means of different resolution levels. Along with this structure, the pgPointCloud extension is used to store each node in the octree as a single entity in the database. A new table has been added to the database schema that stores the content of each node in the octree. This way, each node is referenced by a feature id that is linked to a so called pc_patch which contains the point cloud data for the node. The import to and the export from the database is handled by the so-called "Point Cloud Data Abstract Library" Each feature_id is assigned an ending that corresponds to the octree address invented by the Potree data structure.

For storing and loading the LAS/LAZ file, we rely on the PDAL library. This library offers a comprehensive set of functionality for translating and manipulating point cloud data.

The client implementation for lidar point clouds is a Web viewer for large point clouds that uses standard web technologies and requires no additional plugins. The client is a modified version of the Potree client framework, which is a free and open source WebGL-based viewer for large point clouds. It is based on the octree data format described in the previous section. The use of this multi-resolution enabled data structure allows the client to cull away data and to only load what is visible within the rendered scene. This approach significantly reduces required bandwidth as well as the amount of data to be cached and handled by the client.

As mentioned in the previous section, the developed client is based on the existing Potree implementation. In order to make use of the developed streaming capabilities of the SOS, we have replaced the general file-based loading mechanism with an additional adapter for the SOS interface. Initial information such as service metadata and detailed descriptions of the point cloud contents (i.e. offerings) that are available on the SOS service, are received by calling the GetCapabilities operation. For each offering in the GetCapabilities response there exists a point cloud stored in the octree data format on side of the SOS. The client takes this information and provides the selection capabilities to the user.

Once a specific point cloud (offering) has been selected by the user, the client starts retrieving additional information about the requested point cloud. Therefore, it uses the GetFeatureOfInterest operation to request a listing of all octree nodes related to the selected offering. When the GetFeaturesOfInterest response is retrieved by the client, the client starts loading the scene by first requesting the root element r of the octree and then requesting its child elements r0 to r7. Based on the users view frustum in the client, the client decides on its own, which parts of the octree gets requested from the SOS and rendered. For the requests to download the different chunks of the LiDAR data set it makes use of the GetResult operation provided by the SOS.

Each GetResult response that is retrieved by the client holds the information of the requested node element as Base64 decoded LAS or LAZ. To make use of the encoded content, the client first decodes the resultValue element of the response back to bytes and then interprets the binary content as LAS or LAZ.

The SOS interface is suited for serving LiDAR data in a JPIP-like streaming approach.

The requesting of individual chunks of a LiDAR scene can be achieved through the GetResult operation. A requirement for applying this approach is the pre-processing of the LiDAR scene to be published so that it is subdivided in patches with different spatial extent and resolution. Subsequently it is possible to treat every patch as a distinct feature which can be combined by the client with other data chunks.

Further SOS operations needed for this approach comprise:

As the ResultHandling operations are not a mandatory element required by the SOS standard, they should be made mandatory by a dedicated SOS 2.0 LiDAR Streaming Profile.

The Sensor Web data models and formats can cover the requirements for streaming LiDAR data. However, it is recommended for the future to support native (binary) data formats for LiDAR data as output of the GetResult operation.

The performance observed during the implementation and testing was suitable for the purposes of the testbed (loading chunks of the LiDAR scene within a few seconds). However, a certain delay was induced by the need to en-/de-code the binary LiDAR data into a text-based XML document. By avoiding this overhead, it would be possible to retrieve the requested data chunks from the database in order to write it directly into an output stream. We expect that removing this bottleneck by allowing binary GetResult response contents, a significant performance improvement would be achieved.

Compared with other data streaming approaches such as JPIP, the data streaming through the SOS interface as it was developed in Testbed-12 has a few differences:

In summary, the LiDAR SOS requires slightly more business logic on the client side compared to other approaches. However, during the implementation of the LiDAR SOS client, it became clear, that this additional logic can be implemented in a straightforward manner with reasonable efforts.

As a final statement, it is possible to conclude that the SOS 2.0 Standard is suitable for enabling the streaming of LiDAR data. Based on pre-processing of the LiDAR data sets that shall be published and the SOS result handling operations it is possible to subset a LiDAR scene into smaller chunks that can be step-wise loaded by a client. It has to be noted, that this mechanism requires a certain amount of business logic on the client side but as demonstrated by the LiDAR SOS client, the implementation is feasible with reasonable efforts.

Recommendations on the improvement of the underlying standards comprise especially the idea to allow the GetResult operation responses to contain binary data without the requirement to embed it into an XML container. This would avoid an additional en-/de-coding step during data transmission which would increase performance.

Another topic would be to develop a dedicated SensorML profile for the specific metadata of LiDAR data. While the encoding of the necessary metadata is generally feasible, formal guidance how to achieve this would further increase interoperability.

As a final step of future work, these recommendation could result in a dedicated SOS 2.0 Profile for serving LiDAR data (e.g. including the ResultHandling operations which should be made mandatory for those SOS servers that shall server LiDAR data).