6.1.1.  Tracking of Buses

In the framework of the Testbed-17 MF thread, a use case was selected where trajectories are represented by school buses at a school parking lot while students are getting on or off the buses.

The project’s raw data was composed of two videos of the same scenario, one recorded with a drone and the other one recorded with a GoPro action camera. The input data for the Machine Analytics Client was the output data from the Tracking Service which was stored and provided by the storage service.

The steps in this scenario included:

1. Detect objects in each frame and track them in consecutive frames.

2. Transform objects’ locations from the video space to geographic space.

3. Link object detections in consecutive frames as a single moving object. Smaller incomplete series of detections are classified as tracklets, while a complete series of detections are classified as the track.

4. Send collected moving object’s data to a storage service.

5. Analyze data retrieved from the storage service to generate information such as moving feature distribution, average trajectory time, dwell time, detection of moving feature clusters and detection of trajectory clusters.

6.1.2.  Autonomous vehicle use case

Following on from work in the Testbed-16 initiative, Web Video Map Track (WebVMT) has been used to previzualize and analyze multi-sensor data from an autonomous vehicle in the Testbed-17 initiative.

Ordnance Survey provided video and lidar data from a StreetDrone vehicle navigating the roads around their headquarters in Southampton, UK. The footage was shot from a front-facing camera on the vehicle and shows a number of moving objects including a cyclist, pedestrians and other road vehicles as well as static objects such as hedges, road signs and lamp posts.

1. Tracking Vehicles: If we could detect and track the oncoming vehicles in the StreetDrone videos, this would produce a set of short tracklets which correspond to different moving (or stationary) features. Being able to track the stationary cars in the car park would also be useful to classify them correctly as parked vehicles, which is easy for humans but trickier for a computer as it’s the camera that’s moving rather than the vehicle itself. Tracking moving features from a moving camera is a particularly useful capability for autonomous vehicles and dashcams.

2. Tracking Dangers: If we could detect and track the cyclist in the second StreetDrone video, this would produce longer tracklets as the bike stays in shot for a while, though they pass behind a couple of lamp posts around 0:50. There’s also a pedestrian at about 1:25. Being able to track cyclists and pedestrians would also be a useful capability for autonomous vehicles and dashcams.

3. Associating Views: We could extend these ideas to multiple cameras by using the BikeBro video which includes two views and avoids synchronization issues by baking both into a single video frame. If we could detect and track moving vehicles from the front and rear views, the resulting tracklets could be (trivially) paired together to form longer tracks for the passing vehicles. This capability enables autonomous vehicles to track other vehicles moving around them on roads with multiple lanes as the same vehicle could appear in different camera views at different times and differ in shape due to the different view angles.

All of these capabilities are also useful for the traffic camera use case, which would enable roadside camera footage to be aggregated with dashcam video to track vehicle movements.

6.1.3.  Windsor Scenario

6.2.1.  Storage service

The storage service receives and returns moving features as JSON objects. The current format does not conform to OGC Moving Features JSON encoding (OGC 16-140r1), but this shortcoming is due to a lack of time in implementation development rather than a problem with the JSON encoding.

JSON objects are not stored “as-is”. Instead, the objects are deconstructed. Each feature type is represented by a database table and each property such as start_time, duration, etc. is extracted and stored in a column of the feature table. This data organization is described in OGC 06-104 (Simple Features SQL). An inconvenience is that all features and properties must be known in advance since they are hard-coded both in the Java code reading JSON objects and in the database schema (note: a dynamic approach is possible but has not been implemented in this testbed). But advantages of this approach are that:

• A database index can be applied on those property values,

• The full power of SQL expressions is available for data manipulation,

• Data can more easily be exported to other formats such as netCDF (OGC 16-114r3).

For this testbed, a PostgreSQL database was used. The bus coordinates were stored in a geometry column using the PostGIS extension. It would have been possible to go one step further by using the MobilityDB extension, which would allow storing the “moving geometries” instead of the static geometries in a PostgreSQL database, however, that moved to future development.

Queries can be done using either an implementation of OGC API — Features (OGC 17-069r3) or by using a Common Query Language (CQL) query. CQL supports the use of more advanced filtering conditions such as “features at a distance closer than X from geometry Y”. The storage service can parse CQL and translate it to a mix of SQL and Java code. The SQL statement uses instructions such as ST_Within or ST_Intersects. A spatial database will use its index for efficient execution of those statements. If some parts of the query cannot be translated to SQL, then the storage service will execute them in Java code on the remaining features after the initial filtering done by the database.

The set of operations defined by the OGC/ISO Filter Encoding standard (ISO 19143) and SQLMM standard assumes static geometries. For example, the above-cited example “features at a distance closer than X from geometry Y” does not take time into account. If geometries are trajectories, that operation will check the shortest distance between trajectories regardless of whether the moving features were at those locations at the same time or not. The ISO 19141 (Moving Features) standard defines new operators such as nearestApproach which could be used in CQL expressions. There is no OGC standard defining a filter encoding for moving features operations, but this testbed explored what they may look like (without reaching the point of experimenting with it in the storage implementation).

9.1.1.  Geometry (ISO 19107)

The ISO 19107, Geographic information — Spatial schema standard defines a GM_Object base type which is the root of all geometric objects. A GM_Object instance can be regarded as an infinite set of points in a particular coordinate reference system. The GM_Object base type has various subtypes such as GM_Point, GM_Curve, GM_Surface and GM_Solid. The UML below shows the GM_Object base type with its operations, for example distance(…) for computing the distance between two geometries. All those operations assume static objects, without time-varying coordinates or attributes. For example the distance(…) operator does not take in account whether the two objects were at the same time at the location where shortest distance is found.

Figure 5 — GM_Object from ISO 19107:2003

Geometry, topology and temporal-objects (GM_Object, TP_Object, TM_Object) are not features. These types can provide types for feature properties, but cannot be specialized to feature types. Feature types and properties are described in the next section.

9.1.2.  Features (ISO 19109)

The ISO 19109, Geographic information — Rules for application schema standard defines types for the definition of features. A feature is an abstraction of a real-world phenomenon. The terms “feature type” and “feature instance” are used to separate the following concepts of “feature”:

Feature type

The whole collection of real-world phenomena classified in a concept. For example, the “bridge” feature type is the abstraction of the collection of all real-world phenomena that is classified into the concept behind the term “bridge”.

Feature instance

A certain occurrence of a feature type. For example, “Tower Bridge” feature instance is the abstraction of a certain real-world bridge in London.

Feature type instance

The UML shown below contains a subtlety explained here but ignored for simplicity in the rest of this document: FeatureType is defined as a metaclass, i.e. a class for describing other classes. A feature type instance (not to be confused with a feature instance) is a class that represents an individual feature type. For example, “bridge” and “park” are two instances of FeatureType. Then “Tower Bridge” and “Golden Gate Bridge” are two Feature instances of the “bridge” FeatureType instance, and “Jardin des Tuileries” is a Feature instance of the “park” FeatureType instance.

The UML class diagram in Figure 6 shows the ISO 19109 General Feature Model. A FeatureType contains the list of properties (attributes, associations and operations) that Feature instances of that type can contain. In other words, a feature can be seen as a collection of property values. Geometries are feature properties (more precisely attributes) like any other, without any special treatment. All properties shown below are assumed static, without time-varying values.

Figure 6 — General Feature Model from ISO 19109:2009