Testbed-12 JSON and GeoJSON User Guide

JSON is a very simple data model that can represent four primitives (strings, numbers, booleans (true or false) and null) and includes two structured types (objects and arrays). Strings are enclosed in quotation marks, numbers does not have quotation marks, objects are enclosed in curly brackets "{}" and a arrays are enclosed in square brackets "[]". An object is an unordered collection of zero or more parameters (name and value pairs, separated by a colon), where a name is a string and a value is a primitive or a structured type. Parameters are separated by commas and usually each of them is written in a different line. An array is an ordered sequence of zero or more values (primitives or structured types, separated by commas).

The fact that the names of properties are enclosed in quotation marks and the use of ":" are marks of identity that helps to visually identify that text documents are actually instances written in JSON (instead of e.g. C++ or Java data structures).

In the example, a picture is represented as an object with two numerical properties ("width" and "height"), a string property ("title"), an object property ("thumbnail"), a boolean property ("animated") and an array of numbers property ("ids"). Nothing in the document indicates that it is describing a picture. The reason is that we are supposed to assign the data structure to a variable the name of which will tell us what the variable is about.

Check on KVP_GET_client_request for the way to incorporate a JSON document into a JavaScript code on the fly.

In the web, we find several comparisons between XML and JSON, some of them trying to do statistical analysis on some criteria, such as verbosity or performance. Some others (many, actually) are more based on opinions than in facts. This document will try to escape this debate and focus on practical facts.

XML was designed by a consensus process in several Internet groups and became a W3C recommendation on February 10th, 1998 as a document standard completely independent from any programming language. Since then, hundreds of document formats based on XML syntax had been proposed, including RSS, Atom, SOAP, and XHTML, Office Open XML, OpenOffice.org, Microsoft .NET Framework and many others. OGC has adopted XML for most of its web service messages and for several data formats, including GML, WaterML, SensorML, GUF, etc. XML has some interesting additional components: XML Schema/RelaxNG/Schematon provide ways to restrict the classes and data types to a controlled set of them. Actually, all document formats cited before provide a some form of schema document that accompanies the standard document. By providing schema files, standards incorporate a very simple way to check if a document follows the standard data model: a process executed by automatic tools that is called "XML validation". Other components of XML family are XPath (a query language), XSL transformations, etc. With time, these components have been implemented in many programming languages and tools.

JSON history is completely different. JSON was introduced in 1999 as a subset of the JavaScript language that extracted the essentials for defining data structures. This original idea is stated in the RFC7159: "JSON’s design goals were to be minimal, portable, textual, and a subset of JavaScript". After 2005, JSON became popular and is used in the APIs of many web services offered by big companies on the web, such as Yahoo or Google. Currently, JSON is no longer restricted to web browsers, because JavaScript can now be used in web services and in stand alone applications (e.g. using node.js) and also because there are libraries that can read and write JSON for almost every programming language (see a long list of programming languages that have some sort of JSON libraries on json.org).

The basics of JSON encoding are defined initially in the RFC4627 (2006) and more recently in the RFC7159 (2013).

The following aspects have been extracted from the RFC7159 and, in the opinion of the authors of this document, should be considered requirements to JSON in OGC implementations of JSON encodings. These aspects go a bit beyond the immediate description of a simple JSON format but are needed to improve JSON interoperability. In the following text, the use of quotation marks indicates that the text comes directly from RFC7159 (sometimes with minimal editorial changes).

In addition to these requirements, the authors consider that adding an extra requirement to normal behavior for JSON parsers is interesting: * A parser that finds an object property that it is not able to recognize, should ignore it rather than treating it as an error. (This is not what happens by default in the case of XML validation). In special circumstances JSON can limit this inherent behavior, if conveniently justified, but not in general.

To increase interoperability the authors of this document would like to add considerations for including more simple data types as restrictions of the "string" type. They were taken directly from the "format" parameter of the JSON Schema standard candidate:

See more detail of these four simple data types below.

In addition to these simple types, complex geometrical types are fundamental for OGC and will be discussed in Geospatial_dimension_in_JSON.

OGC is transitioning from standards that were written in plain English to a robust way of written standards based on requirements classes that are linked to conformance test classes. Conformance tests are designed to determine if implementations follow the standard. When an XML encoding is involved, standards that provide XML Schema files defining each data type, provide a straightforward way to check if a document follows the standard: validating the XML document with XSD, RelaxNG or Schematron (or a combination of them).

If OGC is going to adopt JSON as an alternative encoding for data models, some automatic way of validating if objects in the JSON file follow the data models proposed by the corresponding standard could also be convenient.

JSON Schema is intended for validation and documentation of data models. It acts in a similar way to XSD for an XML file. Indeed, some applications (such us XML Validator Buddy) are able to combine a JSON file with its corresponding JSON schema to test and validate if the content of the JSON file corresponds to the expected data model. Several implementations of JSON schema validation are available to also be used on-line. An example is the one available in this URL http://json-schema-validator.herokuapp.com/ and the corresponding opensource code available in github https://github.com/daveclayton/json-schema-validator.

The number of aspects that JSON Schema can validate is lower than the ones that XML Schema can control. Some factors contribute to that:

Unfortunately, JSON schema is a IETF draft that expired in August 2013 and the future of the specification was uncertain. One of the authors blogged that he is forced to abandon the project due to lack of time. The project has been reactivated in September 2016 and a new version of the IETF documents has been released with minimum changes. New releases with descend compatibility have been promised.

Note that the Internet media type is "application/schema+json". According to the last available draft of JSON Schema (v4), there is not a new file extension proposed for files storing JSON Schemas. The file extension ".json" is used profusely. To make the situation a bit more complex, there is no documented mechanism to associate a JSON instance to its schema (even if it seems that some applications use "$schema" to do this; as discussed in https://groups.google.com/forum/#!topic/json-schema/VBRZE3_GvbQ). In preparing these examples, we found the need to be able to prepare json instances and json schemas with similar file names to make the relation between them more explicit and it was practical to name the schema files ending with "_schema.json".

Lets use a simple feature example encoded in JSON to later illustrate how JSON Schema is useful for documentation and validation.

Now let’s define a JSON Schema for validating it. The first thing we need is to start a JSON file with an indication telling everybody that this is a JSON Schema by adding a "$schema" property in the root object of the schema. The value used in this examples reflects the last draft version available some months ago (i.e. v4).

Title and description are useful properties to describe the schema purpose and the objects and properties it will validate.

The root element can be an object or an array. In this case we are validating an object.

Now it is time to enumerate the properties. The properties array allows to enumerate the property names and to list their attributes. In the next example, there is only one property that is called "river". This property is an object and is declared as required.

Since river is an object, we can repeat the previous pattern for it. In particular, a river object has a name and this name is an "string".

A river has additional properties and some of them are numeric. Please note that in the case of numeric properties, the numeric allowed range can be indicated using minimum and maximum. In this case, we are forcing numbers to be non-negative since they represent characteristics that cannot be negative.

Now we add a river property that is called bridges and that can contain a list of bridge names. It is encoded as an array of strings.

Finally, we could use one of the JSON online schema validator tools to check the validity of the previous JSON file. There are many online validators and the initial JSON example has been validated with the proposed JSON Schema with the following validators:

If we simply change the length of the river to a negative number (e.g. -1) we will get an error report that varies in the text from one implementation to the other but all give us an indication of the problem:

Lets consider now that I need to encode rivers and lakes. In this case, we will need an object that can present itself either as a river or as a lake. We have already seen an example for a river, and we now present an instance for a lake.

Obviously, rivers and lakes will have different properties. There is a oneOf property in JSON Schema that allows a thing to present more than one alternative definition. This way both, the previous JSON instance for the river and the one in this subsection, will be validated with the same JSON Schema.

After showing how to do a single feature (i.e. rivers and lakes, each one in an independent JSON document that can be validated with the same JSON Schema) to show how to represent a feature collection as arrays can be useful. Following this approach, we are able to include rivers and lakes as array items in the same JSON file:

This can be validated by the following JSON Schema, that is very similar to the last one, but defines the root element as an array of items.

JSON is one of these cases where simplicity is highly appreciated. It could be useful to consider a second alternative, where there is no need to use an object name. Instead we will use a "type" property to differentiate among object types and this will result in a notation with less indentations. Apart from being more elegant (what is a matter of opinion) it will result in a much more nice conversion to RDF when JSON-LD @context is introduced later (see Apply_JSONLD_to_JSON_objects_subsection).

This is the corresponding JSON Schema that can be used to validate the array. Note that only "river" and "lake" values are allowed in the "type" key, and any other value will generate a validation error.

In JSON Schema, one can do much more than what has been explained here. Most of the needed characteristics of UML class diagram usually included in OGC and ISO standards, such as, generalization, association, composition, etc can be implemented by JSON Schemas as comprehensively discussed in the OGC 16-051 Testbed 12 A005-2 Javascript JSON JSON-LD ER.

Following the RFC5988 description, a link is a typed connection between two resources that are identified by internationalized Resource Identifiers (IRIs) [RFC3987], and is comprised of:

This schema was adopted both by Xlink and Atom in a similar way. The question that is discussed in this section is what is the status of links in JSON and what is the recommendation for the OGC.

In the next section we will explain JSON-LD in detail (please see Apply_JSONLD_to_JSON_objects_subsection) but, in this subsection, we can anticipate what is essential in JSON-LD that allow it to work with links. In JSON-LD, objects have an id that identifies the context IRI. To do that, JSON objects include a property called @id. As we know, a JSON object has properties, and JSON-LD defines how to use a key-value property in a particular way that allows expressing a link from this object to other objects.

Adding JSON-LD to a JSON file introduces the possibility to define links as a property name that has a semantics defined by IANA (such as "describedBy") and the value of the property that has an @id type and shall contain a URI. This is done in a special @context property.

Both JSON fragments can be combined in a single JSON-LD document

JSON-LD allows for a more elegant notation that defines abbreviated namespaces that can be reused later.

JSON-LD can be automatically converted to RDF triples. In the conversion, done using the JSON-LD playground, the second triple expresses that the dbpedia id (the context IRI) is describedBy (the link relation type) a wikipedia URL (the target IRI).

Hypertext Application Language (HAL) is a simple format that tries to give a consistent and easy way to express hyperlinks between resources. It was defined by Mike Kelly, who is a software engineer from the UK that runs an API consultancy helping companies design and build APIs.

The book "RESTful Web Services Cookbook; Solutions for Improving Scalability and Simplicity" By Subbu Allamaraju, in its chapter "How to Use Links in JSON Representations" proposes a direct translation of the Atom xlink notation in two forms:

This approach is consistent with what is proposed and generalized for applying it in JSON Schema: JSON Hyper-Schema: A Vocabulary for Hypermedia Annotation of JSON.

There is also a more compact format alternative.

The later alternative has the advantage that checking for a the presence of a "describedBy" linking is easier in JavaScript and at the same time looks almost identical to the HAL proposal.

To do the same with the first alternative a JavaScript loop checking all links until finding one of the describedBy type will be needed.

If we remove the grouping property "links", then we almost converge to the JSON-LD alternative.

Even if it is difficult to formulate a recommendation, the authors of this guide consider that the JSON-LD alternative has the advantage of simplicity and, at the same time, is the only alternative ratified and approved by a standard body. It has also the advantage to connect with the RDF world.

JSON-LD is a lightweight syntax to encode Linked Data in JSON. Its design allows existing JSON to be interpreted as Linked Data with minimal changes. JSON-LD is 100% compatible with JSON. JSON-LD introduces a universal identifier mechanism for JSON via the use of URIs, a way to associate data types with values.

JSON-LD is considered another RDF encoding, for use with other Linked Data technologies like SparQL. Tools are available to transform JSON-LD into other RDF encodings like Turtle (such as the JSON-LD playground).

The authors of this document perceive the current documentation of JSON-LD as confusing. That is why another approach in explaining how to use JSON-LD is exposed here.

The main objective of JSON-LD is to define object identifiers and data types identifiers, on top of JSON objects. The identifiers used for objects and data types are unique URIs that provide extra semantics because they reuse definitions on the web (semantic web).

First of all, JSON-LD defined two specific properties for each object: @id and @type that can be populated with the identifier of the object and the complex data type identifier of the JSON object. Please note that even if JSON (and javascript) does NOT provide classes, in JSON-LD we assign a class name (a.k.a. data type) to an object. In this example we start by defining the Mississippi river.

The conversion to RDF results in a single triple stating that the Mississippi river is of a river type.

To add a property to the Mississippi river we should define the semantics of the property by associating a URI to it. To do this we need to add also a @context property. Here we have two possibilities:

In this example we are adding a name to a river resulting on a second triple associated to the object id. In this case we reuse the schema.org vocabulary to define the semantics of the word name. Note that http://schema.org/ is actually a URL to a JSON-LD @context document that is in the root of the web server defining the actual and complete schema.org vocabulary.

As a second alternative, when the appropriate vocabulary in the JSON-LD format is not available, we can define the needed term on-the-fly as embedded content in the @context section of the JSON instance.

It is also possible to combine two vocabularies, one preexisting and another embedded. This could be particularly useful if, in the future, OWS common decides to release a vocabulary for OGC and OGC concrete services need to extend it. Note that, in this case, @context is defined as an array of an external vocabulary and an internal enumeration of property definitions (in the following example an enumeration of one element).

Now we can refine the example and provide a more elegant encoding introducing the definition of abbreviated namespaces and their equivalent URI namespace.

By default, JSON-LD considers properties as strings. JSON-LD also permits to define data types not only for the objects but also for individual properties. It is common to define numeric data types.

An interesting aspect of JSON-LD is that it overwrites the behavior of JSON arrays. In JSON, arrays of values are sorted lists but in JSON-LD arrays are sets with no order. This way, in the previous examples, bridges is an array but the conversion to RDF is done in a way that "Eads Bridge" and "Chain of Rocks Bridge" are associated with the Mississippi river with no order. In general, this is not a problem because most arrays are only sets of values. Nevertheless, sometimes order is important for example in list of coordinates representing a line or a polygon border (imagine what could happen if only one coordinate is out or order!!). Fortunately, there is a way to declare that the array values order is important: using "@container": "@list".

Please note that lists of lists are not allowed in JSON-LD making impossible to transform bi-dimensional arrays of coordinates. This issue is being discussed in Geospatial_dimension_in_JSON.

An special kind of data type is "@id". This indicates that a property points to another object id that can be in the same document or elsewhere in the linked data web. This is the way that JSON-LD is able to define links between objects as previously discussed in JSON-LD_links_subsection.

The ISO 19109 General Feature Model discusses aspects of defining features. The ISO 19109 is generally accepted by the OGC community that includes many of its concepts in the OGC 08-126 The OpenGIS® Abstract Specification Topic 5: Features.

The next figure describes the most abstract level of defining and structuring geographic data. In the context of a geographic application, the real world phenomena are classified into feature types that share the same list of attribute types. This means that if, for example, the geographical application is capturing protected areas, a protected area feature type will define the attributes to capture it and all protected areas will share the same data structure.

In practice, and following the same example, this means that there will be a feature catalogue where an abstract protected area is defined as having a multi-polygon, a list of ecosystem types, a list of ecosystem services, a elevation range, a year of definition and the figure of protection used, etc.

This feature type will be formalized in an application schema. Here, we present a table as a formal way to define the attributes of the protected areas feature type.

This way of defining features is basic for the OGC community. GML has included the concept of the application schema from its earlier versions (i.e. an XML Schema). Nevertheless, there are formats that do not follow explicitly the same approach. For example, GeoRSS uses a fixed structure for attributes (common for all features; whatever the feature type) and adds a geometry. KML did not included the capacity to group features in features types until version 2.2 (the first OGC adopted community standard), and this version 2 is the first one to allow more that one property per feature. It includes a <Schema> tag to define feature types and its property names in a section of the document. Later, the feature type names can be used in PlaceMarks as part of the "ExtendedData/SchemaData" tag.

In the next subsections we will see how JSON can be used in different ways, some of them being compliant to the ISO General Feature Model.

After years of discussion, in August 2016 the IETF RFC7946 was released, describing the GeoJSON format. GeoJSON is self-defined as "a geospatial data interchange format based on JSON. It defines several types of JSON objects and the manner in which they are combined to represent data about geographic features, their properties, and their spatial extents."

It defines the following object types "Feature", "FeatureCollection, "Point", "MultiPoint", "LineString", "MultiLineString", "Polygon", "MultiPolygon", and "GeometryCollection".

GeoJSON presents some contradictions about complex data types: JSON has no object type concept but GeoJSON includes a "type" property in each object it defines, to declare the type of the object. In contrast, GeoJSON does not include the concept of feature type, in the GFM sense, as will be discussed later.

GeoJSON presents a feature collection of individual features. Each Feature has, at least 3 "attributes": a fixed value "type" ("type":"Feature"), a "geometry" and a "properties". Geometry only has 2 "attributes": "type" and "coordinates":

There is no specification on what "properties" can contain so implementors are free to provide feature description composed by several attributes in it.

In GeoJSON:

In practice, this means that GeoJSON can only be used in similar circumstances where KML can be used (but without symbology). GeoJSON cannot be used in the following use cases:

In these cases there are three possible options:

Lets explore these possibilities one by one.

In JavaScript, dates are managed by objects created by with "new Date()".

The list of methods that an object created with new Date() can be used to deal with dates is described here (http://www.w3schools.com/jsref/jsref_obj_date.asp). In JSON we can represent object properties but methods are not allowed for security reasons, so the use of the Date() methods is excluded. JSON does not provide an alternative to dates this leaving a gap in the JSON data types. We have two alternatives: define a JSON date object pattern or use strings that represent dates.

Again we can describe two possible alternatives, complex date type intervals or two strings in an array.

This subsection describes the sequence of steps that client and server have to execute to use JSON in both a KVP HTTP GET and HTTP POST with JSON in the body of the request.