6.1.1.  Tracking of Buses

In the framework of the Testbed-17 MF thread, a use case was selected where trajectories are represented by school buses at a school parking lot while students are getting on or off the buses.

The project’s raw data was composed of two videos of the same scenario, one recorded with a drone and the other one recorded with a GoPro action camera. The input data for the Machine Analytics Client was the output data from the Tracking Service which was stored and provided by the storage service.

The steps in this scenario included:

1. Detect objects in each frame and track them in consecutive frames.

2. Transform objects’ locations from the video space to geographic space.

3. Link object detections in consecutive frames as a single moving object. Smaller incomplete series of detections are classified as tracklets, while a complete series of detections are classified as the track.

4. Send collected moving object’s data to a storage service.

5. Analyze data retrieved from the storage service to generate information such as moving feature distribution, average trajectory time, dwell time, detection of moving feature clusters and detection of trajectory clusters.

6.1.2.  Autonomous vehicle use case

Following on from work in the Testbed-16 initiative, Web Video Map Track (WebVMT) has been used to previzualize and analyze multi-sensor data from an autonomous vehicle in the Testbed-17 initiative.

Ordnance Survey provided video and lidar data from a StreetDrone vehicle navigating the roads around their headquarters in Southampton, UK. The footage was shot from a front-facing camera on the vehicle and shows a number of moving objects including a cyclist, pedestrians and other road vehicles as well as static objects such as hedges, road signs and lamp posts.

1. Tracking Vehicles: If we could detect and track the oncoming vehicles in the StreetDrone videos, this would produce a set of short tracklets which correspond to different moving (or stationary) features. Being able to track the stationary cars in the car park would also be useful to classify them correctly as parked vehicles, which is easy for humans but trickier for a computer as it’s the camera that’s moving rather than the vehicle itself. Tracking moving features from a moving camera is a particularly useful capability for autonomous vehicles and dashcams.

2. Tracking Dangers: If we could detect and track the cyclist in the second StreetDrone video, this would produce longer tracklets as the bike stays in shot for a while, though they pass behind a couple of lamp posts around 0:50. There’s also a pedestrian at about 1:25. Being able to track cyclists and pedestrians would also be a useful capability for autonomous vehicles and dashcams.

3. Associating Views: We could extend these ideas to multiple cameras by using the BikeBro video which includes two views and avoids synchronization issues by baking both into a single video frame. If we could detect and track moving vehicles from the front and rear views, the resulting tracklets could be (trivially) paired together to form longer tracks for the passing vehicles. This capability enables autonomous vehicles to track other vehicles moving around them on roads with multiple lanes as the same vehicle could appear in different camera views at different times and differ in shape due to the different view angles.

All of these capabilities are also useful for the traffic camera use case, which would enable roadside camera footage to be aggregated with dashcam video to track vehicle movements.

6.1.3.  Windsor Scenario

6.2.1.  Storage service

The storage service receives and returns moving features as JSON objects. The current format does not conform to OGC Moving Features JSON encoding (OGC 16-140r1), but this shortcoming is due to a lack of time in implementation development rather than a problem with the JSON encoding.

JSON objects are not stored “as-is”. Instead, the objects are deconstructed. Each feature type is represented by a database table and each property such as start_time, duration, etc. is extracted and stored in a column of the feature table. This data organization is described in OGC 06-104 (Simple Features SQL). An inconvenience is that all features and properties must be known in advance since they are hard-coded both in the Java code reading JSON objects and in the database schema (note: a dynamic approach is possible but has not been implemented in this testbed). But advantages of this approach are that:

• A database index can be applied on those property values,

• The full power of SQL expressions is available for data manipulation,

• Data can more easily be exported to other formats such as netCDF (OGC 16-114r3).

For this testbed, a PostgreSQL database was used. The bus coordinates were stored in a geometry column using the PostGIS extension. It would have been possible to go one step further by using the MobilityDB extension, which would allow storing the “moving geometries” instead of the static geometries in a PostgreSQL database, however, that moved to future development.

Queries can be done using either an implementation of OGC API — Features (OGC 17-069r3) or by using a Common Query Language (CQL) query. CQL supports the use of more advanced filtering conditions such as “features at a distance closer than X from geometry Y”. The storage service can parse CQL and translate it to a mix of SQL and Java code. The SQL statement uses instructions such as ST_Within or ST_Intersects. A spatial database will use its index for efficient execution of those statements. If some parts of the query cannot be translated to SQL, then the storage service will execute them in Java code on the remaining features after the initial filtering done by the database.

The set of operations defined by the OGC/ISO Filter Encoding standard (ISO 19143) and SQLMM standard assumes static geometries. For example, the above-cited example “features at a distance closer than X from geometry Y” does not take time into account. If geometries are trajectories, that operation will check the shortest distance between trajectories regardless of whether the moving features were at those locations at the same time or not. The ISO 19141 (Moving Features) standard defines new operators such as nearestApproach which could be used in CQL expressions. There is no OGC standard defining a filter encoding for moving features operations, but this testbed explored what they may look like (without reaching the point of experimenting with it in the storage implementation).

9.1.1.  Geometry (ISO 19107)

The ISO 19107, Geographic information — Spatial schema standard defines a GM_Object base type which is the root of all geometric objects. A GM_Object instance can be regarded as an infinite set of points in a particular coordinate reference system. The GM_Object base type has various subtypes such as GM_Point, GM_Curve, GM_Surface and GM_Solid. The UML below shows the GM_Object base type with its operations, for example distance(…) for computing the distance between two geometries. All those operations assume static objects, without time-varying coordinates or attributes. For example the distance(…) operator does not take in account whether the two objects were at the same time at the location where shortest distance is found.

Figure 5 — GM_Object from ISO 19107:2003

Geometry, topology and temporal-objects (GM_Object, TP_Object, TM_Object) are not features. These types can provide types for feature properties, but cannot be specialized to feature types. Feature types and properties are described in the next section.

9.1.2.  Features (ISO 19109)

The ISO 19109, Geographic information — Rules for application schema standard defines types for the definition of features. A feature is an abstraction of a real-world phenomenon. The terms “feature type” and “feature instance” are used to separate the following concepts of “feature”:

Feature type

The whole collection of real-world phenomena classified in a concept. For example, the “bridge” feature type is the abstraction of the collection of all real-world phenomena that is classified into the concept behind the term “bridge”.

Feature instance

A certain occurrence of a feature type. For example, “Tower Bridge” feature instance is the abstraction of a certain real-world bridge in London.

Feature type instance

The UML shown below contains a subtlety explained here but ignored for simplicity in the rest of this document: FeatureType is defined as a metaclass, i.e. a class for describing other classes. A feature type instance (not to be confused with a feature instance) is a class that represents an individual feature type. For example, “bridge” and “park” are two instances of FeatureType. Then “Tower Bridge” and “Golden Gate Bridge” are two Feature instances of the “bridge” FeatureType instance, and “Jardin des Tuileries” is a Feature instance of the “park” FeatureType instance.

The UML class diagram in Figure 6 shows the ISO 19109 General Feature Model. A FeatureType contains the list of properties (attributes, associations and operations) that Feature instances of that type can contain. In other words, a feature can be seen as a collection of property values. Geometries are feature properties (more precisely attributes) like any other, without any special treatment. All properties shown below are assumed static, without time-varying values.

Figure 6 — General Feature Model from ISO 19109:2009

9.1.2.1.  Usage in Testbed 17

The UML class diagram above defines the GeoAPI interfaces in the org.opengis.feature package (development branch not yet released). The storage service in this testbed uses those interfaces as the “universal” internal representation of features. The service supports the transfer of features between JSON, netCDF and database representations, together with operations on features. Of note, the UML shown in Figure 6 represents meta-classes and not classes, providing information on features in much the same way that metadata describes data.

The following provides a simplified view of a feature in JSON format. The UML model on the right-hand side of Figure 7 describes the JSON on the left-hand side.

Figure 7 — Source JSON data example

The following is an example logical presentation of the feature and its associated attributes. This approach of classifying features provides a method of making a dynamic feature within the data model.

Figure 8 — Logical Representation

9.1.3.  Simple Features SQL (OGC 06-104)

The Simple Feature Access — Part 2: SQL Option standard describes a feature access implementation in SQL based on a profile of ISO 19107. This standard defines feature table as a table where the columns represent feature attributes, and the rows represent feature instances. The geometry of a feature is an ordinary feature attribute, i.e. a single column in the table. In spatial databases, the geometry columns have a geometry type such as ST_Point or ST_Polygon; in databases without spatial support, an alternative approach described by the standard can be used.

9.1.4.  Filter Encoding (ISO 19143)

The ISO 19143, Geographic information — Filter encoding standard (also an OGC Standard) provides types for constructing queries. These objects can be transformed into a SQL “SELECT … FROM … WHERE … ORDER BY …” statement to fetch data stored in a SQL-based relational database. Similarly, the same objects can be transformed into an XQuery expression in order to retrieve data from XML documents. The UML in Figure 9 below shows the objects used for querying a subset based on spatial operations such as “contains” or “intersects”.

Figure 9 — Spatial operators from ISO 19143

All filters defined in ISO 19143 are static. There is no operator taking movement into account. New operators for moving features are proposed in another section of this ER.

In addition to the abstract model, the ISO 19143 standard also defines how to encode the above objects in XML or CQL formats. The abstract model shown in Figure 9 helps developers to identify which concepts are common to all formats, which makes it easier to implement conversions to SQL or XPath.

9.1.5.  OGC API — Features — Part 1 (OGC 17-069r3)

The OGC API — Features — Part 1: Core (OGC 17-069r3) standard specifies the fundamental building blocks for interacting with features using a Web API. This base standard allows a client application to either get all features from a collection available on a server, to filter by feature properties, or to get feature instances by their identifier.

9.1.6.  Features Filtering web API

The OGC API — Features — Part 3: Filtering and the Common Query Language (CQL)_ candidate standard extends the OGC API — Features — Part 1: Core standard with capabilities to support more sophisticated queries. The conceptual model is close to ISO 19143.

9.1.7.  Moving Features (ISO 19141)

The ISO 19141, Geographic information — Schema for moving features standard extends the ISO 19107 spatial schema for addressing features whose locations change over time. Despite the “Moving Features” name, that standard is more about “Moving geometries”. The UML below shows how the MF_Trajectory type extends the “static” types from ISO 19107.

Figure 10 — Trajectory type from ISO 19141

Trajectory inherits some operations shown below. Those operations are in addition to the operations inherited from GM_Object. For example the distance(…) operation from ISO 19107 is now completed by a time-aware nearestApproach(…) operation.

Figure 11 — Temporal geometry from ISO 19141

The ISO 19141 model was not used directly in the Testbed 17 activity. However, the operations defined by the UML class diagram in Figure 11 can be used as a source of definitions for new filter operations that complement the static operations defined by ISO 19141. For example (non-normative):

Figure 12 — Example of filter extension for moving features

Some temporal operators defined by ISO 19141 are reused in the OGC Moving Features Access standard. That standard was defined before the OGC adopted the practices seen in the above-cited OGC API — Features standard.

Consequently, the Web API part of the OGC Moving Features Access standard may need to be refactored to align with current OGC practices. That said, the operators developed in that standard may be reused.

9.1.8.  Moving Features XML encoding (OGC 18-075)

The OGC 18-075 Moving Features Encoding Part I: XML Core standard takes a subset of the ISO 19141 specification and uses it as a basis for encoding an XML document. That standard also completes ISO 19141 by specifying attributes whose values change over time. This extension to the General Feature Model is shown below:

Figure 13 — Dynamic attribute from OGC 18-075

9.1.9.  Moving Features JSON encoding (OGC 19-045)

The OGC 19-045 Moving Features Encoding Extension — JSON standard takes a subset of ISO 19141 standard and encodes it in JSON format. The standard provides various UML diagrams summarizing ISO 19141.

10.1.1.  Overview

Table 1 — Testbed-17 Moving Features Requirements Class

The Moving Features requirements class defines the requirements for a moving feature as evaluated in Testbed 17. A moving feature is a representation, using a local origin and local ordinate vectors, of a geometric object at a given reference time [ISO 19141:2008]

10.1.2.  Information Resources

The two resources defined in this Testbed 17 Requirements Class are summarized in Table 2.

Table 2 — Moving Features Resources

10.1.3.  Resource Features

The MF-API Items query is an OGC API-Features endpoint that may be used to catalog pre-existing moving features. The Features operation returns data which describes the moving features available from this API. If a mFeatureID is not specified, the query will return a list of the available moving features. The list of moving features returned by the response can be limited using the bbox parameter. This behavior is specified in OGC API-Features. All parameters for use with the Items query are defined by OGC API-Features.

type: object
required:
  - type
  - features
properties:
  type:
    type: string
    enum:
      - FeatureCollection
  features:
    type: array
    items:
      $ref: movingFeatureGeoJSON.yaml
  links:
    type: array
    items:
      $ref: link.yaml
  timeStamp:
    type: string
    format: date-time
  numberMatched:
    type: integer
    minimum: 0
  numberReturned:
    type: integer
    minimum: 0

{ "type": "FeatureCollection",
  "numberMatched": 2,
  "numberReturned": 2,
  "links": [
    { "href": "{root}/collections/{collectionId/items",
      "rel": "self",
      "type": "application/geo+json",
      "title": "Collection Moving Features"
    }
  ],
  "features": [
    { "type": "Feature",
      "properties": {
        "Moving_Features_Distribution": [
          { "timestamp": "2012-05-27T04:03:00Z",
            "count": 12
          },
          { "timestamp": "2012-05-27T04:03:30Z",
            "count": 12
          }
        ],
        "Average_Trajectory_Time": {
          "average_time": "00:03:11.742"
        },
        "Dwell_Time": [
          { "moving_feature_id": 158,
            "start_time": "2012-05-27T04:03:00Z",
            "end_time": "2012-05-27T04:05:27Z",
            "duration": "00:02:27",
            "dwell_time": "00:01:06"
          },
          { "moving_feature_id": null,
            "start_time": "2012-05-27T04:07:00Z",
            "end_time": "2012-05-27T04:10:48Z",
            "duration": "00:03:48",
            "dwell_time": "00:01:15."
          }
        ],
        "Moving_Features_Clusters": [
          { "timestamp": "2012-05-13T00:09:18Z",
            "clusters": [
              { "cluster_id": 0,
                "moving_features_ids": [121,155,158]
              },
              { "cluster_id": 1,
                "moving_features_ids": [123,150,151]
              }
            ]
          },
          { "timestamp": "2012-05-13T00:10:18Z",
            "clusters": [
              { "cluster_id": 0,
                "moving_features_ids": [123,155,158]
              },
              { "cluster_id": 1,
                "moving_features_ids": [121,150,151]
              }
            ]
          }
        ],
        "Trajectory_Clusters": [
          { "trajectories_ids": [161,142]
          },
          { "trajectories_ids": [163,145]
          }
        ]
      }
    }
  ]
}

10.1.4.  Resource MovingFeature

type: object
required:
  - id
  - type
  - geometry
  - properties
properties:
  id:
    type: string
  type:
    type: string
    enum:
      - Feature
  geometry:
    $ref: geometryGeoJSON.yaml
  properties:
    type: object
    nullable: true
  bbox:
    type: array
    minItems: 1
    items:
      type: array
      oneOf:
        - minItems: 4
          maxItems: 4
        - minItems: 6
          maxItems: 6
      items:
        type: number
  interval:
    type: array
    minItems: 1
    items:
      type: array
      minItems: 2
      maxItems: 2
      items:
        type: string
        format: date-time
        nullable: true
  links:
    type: array
    items:
      $ref: link.yaml

{
  "type": "FeatureCollection",
  "numberMatched": 3,
  "numberReturned": 3,
  "links": [
    {
      "href": "http://tb17.geomatys.com/API/feature/collections/Observation/items",
      "rel": "self",
      "type": "application/geo+json",
      "title": "this document"
    }
  ],
  "features": [
    {
      "type": "Feature",
      "id": 1,
      "geometry": {
        "type": "Point",
        "coordinates": [
          -116.1608,
          50.0851
        ]
      },
      "properties": {
        "observation_id": 1,
        "tracklet_id": 23,
        "class": "bus",
        "time": "2021-07-29T04:04:03Z",
        "color": [44,33,22],
        "bbox": [944.0,765.0,123.0,59.0]
      }
    },
    {
      "type": "Feature",
      "id": 2,
      "geometry": {
        "type": "Point",
        "coordinates": [
          -117.1608,
          51.0851
        ]
      },
      "properties": {
        "observation_id": 2,
        "tracklet_id": 23,
        "class": "bus",
        "time": "2021-07-29T04:05:03Z",
        "color": [44,33,22],
        "bbox": [944.0,765.0,123.0,59.0]
      }
    }
  ]
}

10.2.1.  HTTP Response

Each HTTP request shall result in a response that meets the following requirement.

Table 7 — General HTTP-Response Requirements Class

The YAML schema for these results is provided in Figure 18.

title: Exception Schema
description: JSON schema for exceptions based on RFC 7807
type: object
required:
  - type
properties:
  type:
    type: string
  title:
    type: string
  status:
    type: integer
  detail:
    type: string
  instance:
    type: string

Figure 18 — HTTP Response Schema

10.2.2.  HTTP Status Codes

Table 8 lists the main HTTP status codes that clients should be prepared to receive. This includes support for specific security schemes or URI redirection. In addition, other error situations may occur in the transport layer outside of the server.

Table 8 — Typical HTTP status codes