OGC Testbed-16: OpenAPI Engineering Report

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

Using UML to model data structures and interface definitions is the original purpose of the MDA approach to architectures. This section does not provide a history of the use of MDA in geospatial technologies as this ER assumes that the reader is familiar with MDA.

This section presents the recent work done within OGC Testbeds and Pilots to provide a grounding for the decisions made in the OpenAPI Thread of Testbed-16.

The UML to GML Application Schema (UGAS) Pilot is the work that directly precedes the Testbed-16 OpenAPI activity. The output of that Pilot was an Engineering Report (https://portal.ogc.org/files/?artifact_id=95469&version=1) and a version of ShapeChange that was used in the OpenAPI thread of Testbed-16.

The UGAS Pilot sought to address four key issues with respect to JSON schemas:

Points 1 and 4 are all relevant to the work done in Testbed-16 and are reviewed in the following paragraphs.

The UML application schema to JSON schema rules are primarily defined to convert application schemas, that is, a data model for a particular application to a JSON representation. The conversion rules are designed for a direct mapping between UML classes to JSON fragments without implementing inheritance in an object-oriented manner. The authors correctly point out that JSON schemas do not directly support the concept of inheritance, which can be managed through a virtual inheritance approach described in the ER. Additionally, as rules are enforced at the supertype level, but not the subtype level, JSON schema does not support class specialization. The approach to class specialization used in this (or UGAS?) was to extend a class data type to and include both in the JSON schema as both are part of the model. A different approach taken is to use a metamodel, model construct where the metamodel is abstract and the model and classes specialized from the generalized metamodel. This ER describes the completed initial work into deriving OpenAPI based interfaces from UML and identifies the building blocks that are required to produce a successful UML model. However, the participants in the UGAS activity did not attempt to produce a UML model to define the interface due to that work item being included in Testbed-16.

In Testbed 14, the participants in the application schema-based ontology development work activity attempted to express Web Ontology Language (OWL) in UML that could then be translated via an MDA process and ShapeChange into an Application Schema. This is a similar problem to the OpenAPI work described in this Testbed 16 ER but with a different target based upon different encoding rules. Essentially, the participants had similar challenges with how to express out of bounds information, much in the same way that conformance classes are used in OpenAPI to describe conformance to clients.

The Testbed-14 participants also wrote an ER that describes conversion of UML application schemas to JSON and JSON-LD (http://docs.opengeospatial.org/per/18-091r2.html). The authors identified several issues with conversion of application schemas to JSON that included: Lack of support of typical object-oriented concepts in JSON such as inheritance, lack of methods to verify schemas in JSON - notably the lack of the ability within JSON to identify its parent schema, and lack of support for other validation mechanisms such as namespaces. The MDA approach requires many of these concepts to either be directly available or to be virtualized. Therefore, many design decisions made in the modeling process were made to make up for these short comings.

One of the recommendations of this ER was to develop an extension to ShapeChange with the JSON schema target. Although this project was not designed to realize that goal, it did provide some lessons learned for would-be implementers that are discussed in a future section. The second recommendation was to develop JSON schemas for ISO schemas. This is a broader question that needs to be better understood. ISO separates their conceptual model standards from their encoding standards (ISO 19115-1:2014 and ISO/TS 19115-3:2016). JSON potentially needs the same treatment, which would have real-world implications for emerging OGC standards such as OGC API - Records as that in development API will serve metadata that is largely ISO derived.

OGC API – Features was the first OGC Web API Standard to be approved that uses OpenAPI to document the API based on the resource paradigm. As OGC API - Features was the first Web API to be approved by the OGC membership, it provides a template for other, new OGC API standards. OGC API - Features has several parts. At the time of Testbed 16, only OGC API - Features - Part 1 was approved. Therefore, Part 1 was considered in this Testbed 16 project as a case study. The structure of the Features API provides a foundation for other standards in the emerging OGC API suite. For example, OGC API - Records has the same structure as OGC API - Features but instead contains conformance classes such as FullTextQuery that allows for operations such as searching metadata records. An additional reason for using API Features is that the draft OGC API - Common uses many of the conformance classes in Features. Therefore, these classes will likely form the basis of the building blocks for future work in the OGC API suite of standards.

The OpenAPI Specification (formerly Swagger Specification) is an API description format for APIs that implement principles of Representational State Transfer (REST). An OpenAPI file allows the developer to describe an entire API including: Available endpoints (/users) and operations on each endpoint (GET /users, POST /users), Operation parameters, and input and output for each operation, and so forth. The use of OpenAPI was experimented with during the development of OGC API Features. Based on that experience, the OGC Membership approved recommending the use of OpenAPI to document any OGC API standard either in development or in the future. OpenAPI provides a well-defined specification for documenting a structure and conformance rules to be adhered to when implementing an API. Additionally, there are many developer tools for generating an API document or stubbing code from an existing API document.

However, OpenAPI is not object-oriented and therefore modeling the OpenAPI specification within the UML class construct requires compensating for this issue. Many of the relationships between OpenAPI objects are straight forward, and can be represented using a simple composition or aggregation depending on the relationship. However, OpenAPI does contain more difficult constructs such as map types (aka dictionary - https://swagger.io/docs/specification/data-models/dictionaries/) in which key/value pairs are both variable. This is slightly more difficult to model in a UML class diagram that relies more upon simple attribute names and variable of fixed type. Additionally, OpenAPI contains arrays of types that need to be ordered, which also are difficult to represent.

Another feature of OpenAPI is that it inherits and extends JSON Schema in an object-oriented manner, but does not explicitly define JSON schema types in the specification. Therefore, a true UML model would also include all of the aspects of JSON schema and all of its extensions. While possible, this was considered out of scope in Testbed 16.

MDA has been a recognized approach to artifact design and generation for decades. One of the motivating principles behind MDA is the approach to design and the ability to remain technology agnostic. MDA involves creating an initial model, called the Platform Independent Model (PIM) and then generating Platform Specific Models (PSM) automatically from the model. This provides several advantages including: Standardized documentation of artifacts, ability to change implemented technologies without changing the underlying model and the ability to extend the underlying model and quickly generate the artifacts required for implementation.

Generation of OGC API standards using the MDA approach is sensible as the new suite of OGC API standards utilize the same underlying model. Changes and extensions to the model are allowed. However, this can be done in a consistent and therefore controlled manner. The modeling language for the MDA approach is often UML class diagrams as they describe the class attributes, relationships and methods. This combination of entities allows the model to express both structure and behavior and is therefore suited to modeling APIs and associated data schemas.

MDA can also be used for interface definitions. However, MDA has also been used for data structure definitions using UML class diagrams. When employed for data definitions, the transformations are rather simplistic as it involves taking the PIM and transforming it into different versions of SQL rather than enforcing encoding rules used for an interface definition.

To transform a UML representation of a model into a configuration file for a target platform, software is used to perform the translation and transformation. ShapeChange was the choice of software for this job in Testbed-16 as it has been utilized in previous Testbeds and Pilots and is mature enough to enable a lot of the functionality required. ShapeChange supports JSON schemas through the work completed in UGAS2020 Pilot and the version used in this project was the development branch taken from the recently concluded UML to GML Pilot as this Testbed thread essentially picks up where that piece of work left off.

ShapeChange connects directly to an Enterprise Architect (EA) model created using the EA (https://sparxsystems.com/) software. Through some configuration, it then takes the UML model and transforms it into ShapeChange XML (SCXML). The SCXML is the starting point for the development effort to convert the UML model into the chosen target, that is, the developer works only on the XML and manipulates it into the format required for the target. The target is then output from ShapeChange according to the rules applied to the SCXML.

As mentioned in the introduction, there is no authoritative UML model for OpenAPI that was suitable for the needs of the project. Therefore, a UML model was created from the OpenAPI documentation. Although the model represents the OpenAPI standard and guidance in the most accurate way possible, there is a trade-off between the following factors:

The UML modeling described and completed in this thread is two-fold:

This approach was chosen to provide future implementers with a metamodel that they could use as a reference for further OGC standard development and documentation.

The OpenAPI metamodel was created from the OpenAPI specification version 3.0.2 and is presented as a set of classes that interact via relationships. OpenAPI is prescriptive about the majority of the OpenAPI objects and relationships. However it is not strictly OO. Therefore, interpretations and design decisions were made to represent OpenAPI as closely as possible whilst adhering to UML convention. An additional factor that influences the design of the model was the requirement to implement a plugin to ShapeChange to produce the OpenAPI document from the UML model using an MDA approach. This has a direct influence on the use of methods within UML, as ShapeChange currently does not recognize methods, therefore the operation call within OpenAPI would naturally sit as a method in a class diagram, but has been included as an attribute due to the shortcomings of ShapeChange. This section shows the views of the UML model as well as some of the design decisions taken.

The metamodel was designed with consistency as a key factor for determining success. For example, the Reference object could be presented as a String type with the name $ref and the value as the variable. From a ShapeChange implementation perspective, this would be the simplest option, as it would follow the typical key/value pairing. However, many reference objects are reused throughout the model and have rules enforced upon them. Therefore, from a UML perspective, implementing a Reference class is prudent so that the specializations of the Reference class can be acted upon accordingly. A similar approach was taken for the Map class, which is discussed later in this section.

The metamodel design process included the entire OpenAPI specification and was done as a reference for implementing classes and as a hook for the ShapeChange implementation. The OGC API - Features - Part 1: Core specialization only uses a subset of the metamodel, but the full model is available in the accompanying Enterprise Architect project.