i. Abstract

The netCDF Moving Features encoding extension is a summary of conventions that supports efficient exchange of simple moving features as binary files. This Discussion Paper is a complement to the Moving Features Encoding Part I: XML Core and an alternative to the Simple Comma Separated Values (CSV) extension. Compared to the CSV encoding, this netCDF encoding offers more compact storage and better performance at the cost of additional restrictions on the kinds of features that can be stored.

ii.          Keywords

The following are keywords to be used by search engines and document catalogues.

ogcdoc, OGC document, Discussion paper, moving features, netCDF, CF convention, trajectory

iii.          Preface

This document does not define any new Moving Features elements, rather it provides a method to encode discrete sampling geometries as netCDF files. This Discussion Paper combines two existing OGC specifications:

•   The Network Common Data Form (netCDF) standard [OGC 10-092r3], which provides the binary encoding of array-oriented scientific data; and
•   A subset of the Climate and Forecast (CF) conventions [OGC 11-165r2], which provides standard names and layout for the arrays to encode in a netCDF file.

This encoding is intended to provide a faster and more compact alternative to the Moving Features CSV encoding using a format readable by existing software.

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

iv.          Submitting organizations

The following organizations submitted this Document to the Open Geospatial Consortium (OGC):

Central Research Laboratory, Hitachi Ltd.
Geomatys

v.          Submitters

All questions regarding this submission should be directed to the editor or the submitters:

1.    Scope

This OGC^® discussion paper gives guidelines for encoding in netCDF files the geometries of features that move as rigid bodies. This is an alternative to the encoding defined by the OGC Moving Features Encoding Extension: Simple Comma Separated Values (CSV) standard. netCDF encoding can store moving features with the following characteristics:

•   Each moving feature can be described with the Schema for Moving Features (ISO 19141:2008);
•   The number of features simultaneously encoded with this format can be massive (many thousands of features);
•   All features can be described using common space-time coordinates;
•   Trajectories are described by a single point (not a line string) at each time step;
•   Foliation order is sequential, i.e., trajectory elements that are parts of one track are ordered by time.

Deforming features such as floodwater and geometric complexes are out of scope.

This OGC document describes relevant attributes from the Climate and Forecast (CF) conventions in sufficient detail that client applications do not need to deal with the entire scope of CF conventions. Applications only need to understand a restricted subset of CF conventions in order to be able to interpret moving features in netCDF files.

2.    Conformance

Conformance with this specification shall be checked using all the relevant tests specified in Annex A (normative).

3.    References

The following normative documents contain provisions that, through reference in this text, constitute provisions of this document. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. For undated references, the latest edition of the normative document referred to applies.

•   OGC: OGC 14-083r2: Moving Features Encoding Part I: XML Core, 2015
•   OGC: OGC 14-084r2: Moving Features Encoding Extension: Simple Comma Separated Values (CSV), 2015
•   OGC: OGC 10-090r3: OGC Network Common Data Form (netCDF): Core Encoding Standard, 2011
•   OGC: OGC 10-092r3: NetCDF Binary Encoding Extension Standard: netCDF Classic and 64-bit Offset Format, 2011
•   OGC: OGC 11-165r2: CF-netCDF3 Data Model Extension standard, 2012
•   Lawrence Livermore National Laboratory: NetCDF CF Metadata Conventions – http://cfconventions.org/
•   ESIP: Attribute Convention for Data Discovery (ACDD) – http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery
•   NOAA: NCEI netCDF Templates – http://www.nodc.noaa.gov/data/formats/netcdf/

The OGC 11-165r2 CF-netCDF standard is based on Climate and Forecast (CF) conventions version 1.6. The Attribute Convention for Data Discovery (ACDD) adds additional attributes that provide useful metadata for Catalog Service on the Web (CSW). The National Centers for Environmental Information (NCEI) document is used as a guideline when an ambiguity needs to be resolved.

This document references OGC 11-165r2 when possible, but the on-line CF, ACDD, or NCEI documents may be referenced when additional information is considered relevant to moving feature binary encoding.

4.    Terms and Definitions

This document uses the terms defined in Sub-clause 5.3 of [OGC 06-121r8], which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this standard.

For the purposes of this document, the following additional terms and definitions apply.

4.1          Moving feature

Representation, using a local origin and local ordinate vectors, of a geometric object at a given reference time [ISO 19141:2008].

4.2          Vector

Quantity having direction as well as magnitude [ISO 19123:2005].

4.3          Geometric object

Spatial object representing a geometric set [ISO 19107:2003].

4.4          Geometric primitive

Geometric object representing a single, connected, homogeneous element of space [ISO 19107:2003].

4.5          Period

1-dimensional geometric primitive representing extent in time [ISO 19141:2008].

4.6          Instant

0-dimensional geometric primitive representing a position in time [ISO 19141:2008].

4.7          Point

0-dimensional geometric primitive representing a position [ISO 19107:2003].

4.8          Trajectory

Path of a moving point described by a one parameter set of points [ISIO19141:2008].

4.9          Foliation

One parameter set of geometries such that each point in the prism of the set is in one and only one trajectory and in one and only one leaf [ISO 19141:2008].

4.10      One parameter set of geometries

Function f from an interval t ∈ [a, b] such that f(t) is a geometry and for each point P ∈ f(a) there is a one parameter set of points (called the trajectory of P) P(t)  : [a, b]  → P(t) such that P(t)  ∈ f(t) [ISO 19141:2008].

4.11      Prism (one parameter set of geometries)

Set of points in the union of the geometries (or the union of the trajectories) of a one parameter set of geometries [ISO 19141:2008].

4.12      Leaf (one parameter set of geometries)

Geometry at a particular value of the parameter [ISO 19141:2008].

4.13      Dataset

An identifiable collection of data [ISO 19101].

4.14      Attribute

NetCDF attributes hold metadata. An attribute contains information about properties of a variable or an entire dataset [OGC 10-090r3].

4.15      Global attribute

Global attributes apply to a whole dataset and may be used to record properties of all the data in a file, such as processing history or conventions used [OGC 10-090r3].

4.16      Variable attribute

Variable attributes record the properties of one variable [OGC 10-090r3].

4.17      Variable

A netCDF variable has a name, type, shape, attributes, and values. In face, much of the netCDF specification consists of defining the specific characteristics of variables. Variables hold data values. In the netCDF model, a variable can hold a multidimensional array of values of the same type [OGC 10-090r3].

4.18      Shape

The shape of a variable is specified with a list of zero or more dimensions [OGC 10-090r3].

4.19      Dimension

NetCDF dimensions are used to specify variable shapes, common grids, and coordinate systems [OGC 10-090r3].

5.    Conventions

This section provides details and examples for conventions used in the document.

5.1    Abbreviated terms

The following symbols and abbreviated terms are used in this discussion paper:

6.    Data model

The data model used by the binary encoding is equivalent to that used by the CSV encoding. Figure 1 shows an example for trajectories of three moving points A, B and C. Each trajectory has the same start time and the same end time.

•   At t = 0 (start of all data), all points start moving.
•   At t = 1, the movement of A is changed.
•   At t = 2 (end of all data), the movement of A, B and C are changed.

In the case, the trajectory of A from t = 0 to t = 1 and from t = 1 to t = 2 is encoded, together with the trajectories of B and C from t = 0 to t = 2.

7.    Data structure and file format

The binary encoding of arrays and attributes is defined by OGC 10-092r3 – NetCDF Binary Encoding Extension Standard: netCDF Classic and 64-bit Offset Format. That netCDF standard defines only a “container” format – it does not define the meaning of attributes stored in the file (the “schema” in XML terminology).

The full content of OGC 10-092r3 standard is needed for Moving Feature Simple Binary; this discussion paper does not propose any addition or subset of OGC 10-092r3.

Meanings of attributes in a netCDF file are defined by a separated standard, OGC 11-165r2 – CF-netCDF3 Data Model Extension. All remaining requirements and recommendations in this Moving Feature Simple Binary document apply to a subset of those CF conventions.

7.1    NetCDF global attributes

Global attributes apply to the netCDF file as a whole. They specify a succinct description of what is in the dataset, the kind of sampling geometry, the geographic bounding box, the vertical extent, the time period, the coordinate reference systems used and many other metadata (see CF convention). Some global attributes are merely a convenience for information that may otherwise be tedious to compute; those attributes are presented in §0.

7.1.1    Non-redundant global attributes

This sub-section presents some global attributes that do not duplicate information provided by other attributes.

7.1.1.1    Identification of conventions

To identify that the file uses the CF convention, OGC 11-162r2 §7.3.2.1 requires that the Conventions global attribute is set to the “CF-1.6” string value. This Discussion Paper uses also some attribute convention for data discovery, which is notified by the “ACDD-1.3” string value.

7.1.1.2    Feature type

The CF conventions allow different kinds of sampling geometry such as point, time series, profile, or trajectory. This binary encoding Discussion Paper focuses on trajectories. This choice constraint the data layout to a series of data points along a path through space with monotonically increasing times (OGC 11-165r2 §7.5.2.3).

7.1.1.3    Title

Both CF convention and ACDD recommend the inclusion of a title in the global attributes.

7.1.2    Convenience global attributes for data discovery

This sub-section presents global attributes that duplicate information available in other attributes, but in a more convenient manner. For example, the geographic bounding box attributes (§7.1.2.1) duplicate information provided by the geospatial bounds attributes (§7.1.2.2 to §0), but may avoid the need to apply an inverse map projection. The geospatial bounds attributes themselves duplicate attributes associated to coordinate variables. But the attributes described in this sub-section make it easier to discover the data. Indeed, all attributes in this sub-section come from the Attribute Convention for Data Discovery (ACDD). By contrast, most attributes in other sections come from the Climate and Forecast (CF) convention.

7.1.2.1    Geographic bounding box

The geographic bounding box is given in longitude and latitude regardless of the Coordinate Reference System (CRS) used by the trajectories. Since those metadata are used for discovery purpose only, the geodetic datum is unspecified. Coordinate precision may be of only two decimal places of a degree.

This geographic bounding box is redundant with the geospatial bounds defined below. The geographic bounding box is optional metadata for easier discovery of a dataset without the need to parse Well Known Text, lookup EPSG codes, and reproject envelopes.

7.1.2.2    Geospatial (not necessarily geographic) bounds

The horizontal part of the spatial boundary of moving features is specified as geometry in Well Known Text (WKT) format. The Coordinate Reference System is specified in §0.

This metadata is equivalent to the mf:STBoundedBy element defined by Moving Features Core and to the @stboundedby element defined by the CSV extension, except that the geometry is not an envelope (but may be an equivalent polygon) and does not contain vertical or temporal components.

7.1.2.3    Vertical bounds

If the feature trajectories are three-dimensional, then the vertical part of the spatial boundary of moving features is specified as a range of floating-point values. The Coordinate Reference System is specified in §0.

This metadata is equivalent to the vertical component of mf:STBoundedBy (Moving Features Core) or @stboundedby (CSV extension).

7.1.2.4    Temporal bounds

The times of the first and last data point in the file are specified as a range of dates. Dates are formatted as specified by ISO 8601:2004, preferably the extended date-time format (YYYY-MM-DDThh:mm:ss optionally followed by the time zone).

7.1.2.5    Spatial Reference System

The Coordinate Reference System (CRS) of the geospatial and vertical bounds (§7.1.2.2 and §0) is specified by an authority code, preferably from the EPSG geodetic dataset. For two-dimensional trajectories, only one CRS should be specified. For three-dimensional trajectories, a single three-dimensional CRS or two distinct CRS (one horizontal and one vertical) may be specified depending on the nature of the vertical heights.

While ACDD uses simple strings of the form “EPSG:4326”, this document recommends URNs of the form “urn:ogc:def:crs:EPSG::4326” instead. However, software should be prepared to read both forms. Note that axis order is the same with both forms, namely (latitude, longitude) in EPSG:4326 case.

This metadata is equivalent to the srsName attribute in Moving Feature core and to the srid attribute in CSV extension, except for the possible separation between a horizontal and a vertical CRS.

7.2    Trajectory lines (data body)

The CF convention proposes four different ways to organize the feature coordinates and attributes in a netCDF file. This best-practice paper chooses the Contiguous Ragged Array representation, which has the following characteristics:

• Can mix short and large features without waste of space;
• Number of points of each feature must be known in advance;
• File can be updated with new features or new points in the last feature (but not new points in previous features); and
• Foliation order is restricted to “sequential”.

The file is organized by first defining netCDF dimensions (not to be confused with spatio-temporal dimensions), then variables. The dimension and variable names shall comply with the restrictions documented in OGC 11-162r2 §7.3.2.1:

7.2.1    NetCDF dimensions

The ragged array representation needs three netCDF dimensions, described below.

7.2.1.1    Maximal length of feature identifiers

If each feature is identified by a string value, then the netCDF file needs to declare the maximal number of characters allowed in those identifiers. Identifiers shorter than the maximal length will be padded by spaces of null characters (readers must be prepared to handle both).

7.2.1.2    Feature instance dimension

The netCDF file shall declare a dimension for information about each feature as a whole (i.e., information that does not depend on the time). The length of this dimension is the maximal number of features that can be stored in the file. It is acceptable to declare a length larger than needed in order to reserve room for future feature additions, provided that values in the count variable (§7.2.2.2) are set to zero for all missing features.

7.2.1.3    Sample dimension

The netCF file shall declare a dimension for the actual data (time, geospatial coordinates and feature attributes). This dimension could have a fixed length, but it is more convenient to declare this dimension length as unlimited if new data need to be appended. Note that a netCDF file can have only one dimension of unlimited length.

7.2.2    NetCDF variables

In the continuous ragged array representation, the netCDF file shall contain the following variables:

• One variable listing feature identifiers;
• One variable counting the number of points in each feature;
• Three or four auxiliary coordinate variables (not to be confused with “simple” coordinate variables): examples: x, y, (z) and t; and
• An arbitrary number of variables for feature attributes (not to be confused with global attributes or variable attributes): examples: state, temperature.

7.2.2.1    Feature identifiers

Each trajectory should have identifying text that specifies the moving feature. The text can be a person ID, a vehicle ID, etc. While the identifier is usually a string, integer types are also allowed.

The data stored in this variable are equivalent to the mfIdRef attribute value in Moving Feature XML file, or to the values in the mfidref column in a Moving Feature CSV file.

7.2.2.2    Count variable

The netCDF file shall contain a variable holding the number of points in each trajectory. The length of this variable is the maximum number of features that the netCDF file can contain.

7.2.2.3    Auxiliary coordinate variables

Trajectory coordinates are specified in one variable for each spatiotemporal dimension. Those “auxiliary coordinate variables” are not subject to the usual restrictions of netCDF “coordinate variables”. In particular:

•   The variable name does not match the dimension name;
•   The values do not need to be ordered monotonically;
•   The variable does not have axis attribute; and
•   The variable may have missing values.

Ordinate values of the first feature – i.e., the feature at index 0 in the feature instance dimension (§7.2.1.2) – are written first. The number of values to write for that first feature is given by the value of the count variable (§7.2.2.2) at index 0. Then the ordinate values of the second feature – i.e., the feature at index 1 in the feature instance dimension – follow. The number of values to write for that second feature is given by the value of the count variable at index 1, and so on.

For example, if a file contains three features identified as “A”, “B” and “C” and if their trajectories are described by the following points:

•   A: start at (11, 2) then move to (12, 3) and finally (10, 3);
•   B: start at (10, 2) then move to (11, 3); and
•   C: start at (12, 1) then move to (10, 2) and finally (11, 3).

Then the feature instance dimension has a length of 3 and the sample dimension (if not unlimited) has a length of 3 + 2 + 3 = 8. In such cases the variables presented in previous sub-sections have the following values:

NetCDF variables having the feature instance dimension (§7.2.1.2):

The auxiliary coordinate variables can be as below:

NetCDF variables having the sample dimension (§7.2.1.3):

In addition to the attributes listed in above requirement, some non-standards attributes may help parsers to identify the coordinate system. In particular, the _CoordinateAxisType attribute may be set to “Lon”, “Lat”, “Height” or “Time” values for a geographic CRS, or to the “GeoX”, “GeoY”, “GeoZ” or “Time” values for other kind of CRS.

7.2.2.4    Feature attribute variables

If the file contains additional attributes for the moving features, they can be declared in array having the same length than the auxiliary coordinate variables. Elements in those variables are mapped to features and time in the same way than the values in auxiliary coordinate variables (§7.2.2.3).

OGC 11-162r2 has additional requirements for the exchange of scientific data.

7.2.2.5    Feature attribute variables as character strings

If the values of a feature attribute variable are character strings, then there is a choice:

1.     If only a limited amount of distinct values are used (e.g., values are members of an enumeration), then those values can be encoded as flags as described in CF conventions §3.5 (details below);

2.     Otherwise a new dimension need to be defined with a length equal to the maximal number of characters to be stored in the feature attribute variable. This dimension is added as the last dimensions of the feature attribute variable, as defined in CF-convention §2.2. This dimension is shown as the “character dimension” in the figure below.

The second approach may result in a netCDF file as big as or bigger than an equivalent CSV file since all character strings shorter than the maximal length are padded with spaces or zero values. For efficiency reasons, this paper recommends the use of flags.