Testbed-12 Semantic Portrayal, Registry and Mediation Engineering Report

The SRIM ontology in Testbed 12 used Dublin Core Metatada Terms. Future work to enhance SRIM is to map it with ISO 19115. Request for changes to improve the current standard ISO 19115 might also be required to better align with Linked Data. ISO 19115 should provide better use of controlled vocabularies, linked data friendly identifiers, and a better service description that enables automated access to services.

In Testbed 12 , the Semantic Registry harvested information from a federation of CSW services to exercise the Semantic Registry Information Model (RIM) and the REST API. Future work could include improving the efficiency of the harvesting process by investigating the publish/subscribe protocol and versioning management of the register items in the Semantic Registry as they change over time.

With the deployment of the Semantic Web and Linked Open Data, data sources have multiplied, as well as, the machine-processable controlled vocabularies that structure and constrain the interpretations of these data. These controlled vocabularies can be ontologies (RDF Schema, OWL), codelists, taxonomies, thesauri (SKOS) sometimes augmented with additional rules (SPARQL/SPIN rules, SWRL, RIF) ,and constraints (SHACL).

Vocabulary directories exist (e.g. LOV), but there is an ever-increasing demand for environments that simplify searching, editing and collaborative contributions to the vocabularies by non-experts of the Semantic Web. This creates a tension between very rich formalisms and a need to democratize participation in the life cycle of controlled vocabularies.

Vocabularies are most likely to be adopted and shared if they are made available easily. Nevertheless, despite successes in the use of SKOS for encoding vocabularies, current standards provide only low-level interfaces to vocabulary data. For example, many vocabularies are published as an RDF document for download. However, if the vocabulary is large, then the download will be commensurately large; if the user only wants to retrieve a single vocabulary term or select a few terms, this option requires processing on the client side. Alternatively, access to vocabularies is often provided at a SPARQL endpoint. SPARQL is the generic RDF query language. While it is powerful, it is also considered a low-level language similar to the relational database query language SQL and normally is only used by database administrators.

Some SKOS vocabularies are published via other HTTP interfaces. However, each implementation uses different protocols and supports a varied set of features (e.g. content-negotiation provided by the GEMET REST interface and NERC Data Grid’s Vocabulary Server SOAP interface). In some cases, one or both of human-readable formats and machine-readable formats are not available. Thus, discovery and access across vocabulary endpoints becomes challenging and ad-hoc.

There is a clear opportunity to design an API to match the SKOS and OWL vocabularies, taking advantage of the fact that most modern vocabulary content is structured using SKOS and OWL classes and predicates. This API can then be used as the basis for various higher-level vocabulary applications (NLP applications, Concept Recommender,Semantic Enricher,etc.) that can be used to enrich for example ISO 19115 metadata and other OGC services using controlled vocabularies in their metadata.

The Testbed 11 and 12 had explored the high-level description of ontologies and schemas to support semantic mediation. Future work can include investigation of the kind of metadata needed to enable search on controlled vocabularies by defining an ontology that addresses the following aspects:

Based on this ontology, we propose to define a standard REST API to search and access vocabulary metadata and their terms using best practices in REST API (hypermedia driven for example).

The Semantic Registry Information Model (SRIM), developed during Testbed 12, is defined as a superset of W3C DCAT standard and encoded as an OWL ontology. However, the OWL does not capture some of the semantic integrity constraints that are necessary to validate the instance information encoded using the SRIM ontology profiles. This is not an isolated problem. The DCAT ontology, for example, defines a set of classes and properties and reuses a large number of external vocabularies such as Dublin Core, but does not provide any restrictions in the ontology. Users have to read the profile documents such as DCAT-AP or GeoDCAT-AP to know which and how properties should be applied for a given class (mandatory, recommended or optional). For example, a Dataset could have only one title per language, or contact information should have either a person name, organization name or position name, and either email or telephone number. These kind of restrictions cannot be captured with OWL, and until now it required human interpretation to implement the constraints in code.

To fill these gaps, the emerging W3C standard called Shape Constraint Language (SHACL) provides a powerful framework to define the "shape" of the graph data and the ability to define complex integrity constraints using well-defined constraints constructs defined in RDF and SPARQL/Javascript constraints. SHACL is not a replacement of RDFS/OWL, but a complementary technology that is not only very expressive but also highly extensible. While RDFS and OWL are used to define vocabularies terms (classes/properties) and their hierarchies (subclasses, subproperties), as well as the nature of the classes and properties (union, intersection, complement of classes, transitive, inverse, symmetric properties, etc.), SHACL is more appropriate to capture the property constraints (cardinality, valid values or shape values and interdependencies between them) and capable of accommodating multiple profiles by providing different shapes for the same ontology. The SHACL vocabulary is not only defined in RDF itself, but the same macro mechanisms can be used by anyone to define new high-level language elements and publish them on the web. This means that SHACL will not only lead to the reuse of data schemas but also to domain-specific constraint languages. Furthermore, SHACL can be used in conjunction with a variety of languages beside SPARQL, including JavaScript. Complex validation constraints can be expressed in JavaScript so that they can be evaluated client-side. In addition, SHACL can be used to generate validation reports for quality control with potentially suggestions to fix validation errors. Overall, SHACL is a future-proof schema language designed for the Web of Data.While SHACL is not yet a standard, there are already existing implementations using it (e.g. Topbraid) .

Future work could include investigation of the use of SHACL shapes to define application profiles, generation and data entry, data validation, and quality control of linked data information.

During the Testbed 11, it was introduced the portrayal ontology that focused on point-based symbology (icons for Emergency Management). Testbed 12 extended this work by providing a richer symbolizer and graphics ontology that can accommodate line and area-based symbols along with graphic attributes applicable to these symbols. Future work could extend the ontology to accommodate more complex symbology including composite symbols and symbol templates. The extended ontology will help describe more advanced symbology standards such as the family of MIL2525 symbols.

This Testbed managed to render symbol legend based on their definition, however more work is needed to develop rendering a SVG map based on the portrayal ontology. Future work can lay out the foundation to express styles that have at least the same expressiveness as SLDs. The proposed work can extend the portrayal ontology to represent composite symbols and symbol templates. Related future work can include investigation of other renderer outputs such as JSON encoding of the portrayal information, so they can be handled on the client side in HTML5 Canvas or other rendering libraries such as D3.js. Other renderers may also investigate SLD production from the RDF descriptors and investigate how unsupported features from the portrayal ontology can be supported in less expressive graphic languages than SVG, such as KML.

Current Catalog solutions are highly dependent upon the metadata model employed for the service and data descriptions. Many of today’s service instances and data holdings are based on an ISO 19115 metadata model using XML as the exchange format. These solutions are very document-centric due to the nature of XML Document Object Model (DOM) that are used to populate these catalogs. This is often a challenge when information need to be integrated and be linked together, as it often requires to use complex protocols (XLink, CSW GetRecords operations) to circumvent the lack of global unique identifier system in XML document. This often leads to significant overhead to query across multiple documents and difficulties in reusing and linking information.

Linked Data provides a solution as information are modeled as a Directed Labeled Graph. The nodes of the graph (called resources) and edges (called properties) are identified with globally unique resource identifiers (URIs) which are assigned to a unique meaning (defined in ontologies). While best practices require to have resolvable resource URIs in Linked Data, in practice it is often not the case. Linked Data from different namespaces are usually aggregated in catalogs to facilitate search and discovery and get easy access to these resource descriptions.

The W3C has released the DCAT recommendation in early 2014. “DCAT is an RDF vocabulary designed to facilitate interoperability between data catalogs published on the Web. By using DCAT to describe datasets in data catalogs, publishers increase discoverability and enable applications easily to consume metadata from multiple catalogs. It further enables decentralized publishing of catalogs and facilitates federated dataset search across sites.” (W3C). Thus, DCAT defines a standard way to publish machine-readable metadata about a dataset. It does not make any assumptions about the format of the datasets described in a catalog.

Since then, a number of profiles of DCAT have been created and adopted such as DCAT-AP, GeoDCAT-AP, ADMS and Project Open Data (POD), mostly focused on Dataset description. There is no well-established profile that either generalize or specialize DCAT to describe services, schemas, schema mapping, vocabularies and portrayal information. The Linked Data Query Language SPARQL and its geospatial extension GeoSPARQL are now well-established standards to query Linked Data representation with geospatial information. The SPARQL query Protocol provides a partial answer to access DCAT-related data but it is not sufficient and sometimes too complex to manage and access catalog information through a REST API. CKAN API, an open-source software powering many Open Data Portal have provided recently a plugin to export data in DCAT format. As there is more and more data published in DCAT format in the community, there is a need to have a simple and standardized REST API to manage and access Linked Data describing assets managed by catalogs/registries.

Furthermore, there are many clients that access web services from web browser using protocol based on JSON written in Javascript. The direct use of RDF formats such as RDF/XML, N3, Turtle has proved that they are not easily exploitable by these clients. The new W3C standard JSON-LD is designed to address this gap. It provides the ability to describe Linked Data in JSON and vice versa by leveraging the JSON-LD Context.

The use of REST API in the web community has been widely successfull because it aligns well with web principles and enables rapid integration to build web-based application. A number of effort related to Linked Data and REST API have been conducted such as Linked Data Platform 1.0 (LDP). Recent trends in the industry see the emergence of Hypermedia formats (HAL, Hydra, Collection+JSON, Siren) to support data REST API with hypermedia controls. This introduces a level of decoupling between server and client ecosystem that could enable future evolution of APIs without breaking client ecosystem. The challenge of this testbed is to find out how REST, Hypermedia API, Linked Data representation and APIs, JSON-LD can be combined together to meet the requirements of the service.

During the OGC Testbed 11, Image Matters LLC developed the first iteration of the Semantic Mediation Service, demonstrating the transformation of Homeland Security Working Group (HSWG) Incident Ontology to the Canadian Emergency Management Symbology (EMS) Ontology, using a rule engine. The engine was based on the Semantic Mediation Ontology and SPARQL Extension ontology expressing rules and functions of transformations between two semantic models (expressed as ontologies). The transformation from a source ontology to target ontology is called Alignment in the Semantic mediation Ontology.

While semantic transformation in the Testbed 11 was demonstrated on Linked Data representation, there is a large number of messages or documents in the current OGC standards that are based on XML representation. Their structures and syntaxes are defined with XML Schemas (GML, ISO 19139, CSGDM, NMIS). The messages often need to be converted to other XML representation based on another XML schema, typically using XSL Transformations but also by scripts. There is a need to search and discover schema definitions and existing schema mappings and finding optimal transformation paths between two schemas and perform these transformations on demand. The current OGC standards do not provide standard profile to represent schemas and schema mappings in existing OGC Catalog Services. Similary, there is no ontology or DCAT profile capable to describe semantically Schema and Schema Mappings.

There is no existing REST service that enables the search and discovery of schemas and schema mapping, performs validation, calculate optimal transformation paths and perform transformation od message from a schema to another.

The current OGC standards related to Portrayal (Style Layer Descriptor and Symbol Encoding) are based on XML schemas technologies. Styles definition, rules, symbolizers and graphics are defined XML document and do not provide global reusable identifier that allow the reusability and linking of portrayal information to emerge a "web of portrayal information". The standard ISO 19117 that describe portrayal information define an abstract model is mostly designed to be implemented in code but is less adequate to be used a descriptive model.

During the testbed 11, an initial set of portrayal ontologies were defined to describe semantically styles, portrayal rules, point-based symbols and graphics. The information were made accessible through a specialized REST API to be accessible by a WPS to produce SLD documents. The ontologies were limited only on point-based graphic symbols (based on PNG and True type font). The ontology was not able to model line, text and area based symbols and more advanced graphic objects and properties.

A number of attempts have been done in previous testbeds (OWS-8) to represent portrayal information in OGC catalog, but no standard profile has been defined so far and none of them has used a semantic-based approach.

There is a need to manage portrayal information in a catalog that can be related other information such as feature types, layers and taxonomies. This information can be searched and used to perform map portrayal using symbologies from different communities. The initial implementation of the Semantic Portrayal Service in the OGC Testbed 11 did not provide rendering endpoint for the symbology.

The following were the initial four general RFP requirements on the DCAT as a REST service implementation.

From these initial high-level requirements, we derived and added the following ones:

The following items describes the requirements for the Semantic Mediation Service:

The following items describes the requirements for the Semantic Mediation Service

The initial requirements of the testbed was to implement a DCAT REST Service that enables the access to DCAT Dataset information. However it quickly emerged that the integration of ISO 19139, NMIS, Services, Schema, Schema Mapping and Portrayal information in the service was not possible using DCAT alone, as DCAT is mostly focused on Dataset description. More generalization was needed to accommodate the different types of information objects in the future (for example Map, Layer, Vocabulary, Sensor).

The closest standard that provides an extensible framework to represent different business objects is the Electronic Business eXtensible Markup Language (ebXML) Registry Information Model (ebRIM). The ebXML registry describes objects that reside in a repository for storage and safekeeping. The information model does not deal with the actual content of the repository. All elements of the information model represent metadata (data type and data relationships) about the content stored in the repository. Such information is used to facilitate ebXML-based Business-to-Business partnerships and transactions. The registry information model provides a high-level schema for the ebXML registry. The ebRIM model has a lot overlapping constructs with the ones defined in RDF and OWL. However we found out that the definition of the classes and properties in ebRIM were often not well aligned with well-established ontologies (with the exception of Dublin Core Terms) and the best practices in the Linked Data community. ebRIM introduces a lot of verbosity to describe simple graph structure thus making very hard to build queries that naturally matches the graph structure. The Linked Data model and the standard SPARQL query language provides a more modern, decentralized approach that lowers the bar of integration and learning curve of manipulating graph-oriented data structure and seems a better match to favor reusability and linkage of information.

By analyzing the ebRIM Model and the ISO 19135 standard, we decided to create a superset of DCAT that support different types of registry items. We borrowed the notion of registers and items from both standards. The proposed solution is a new service called Semantic Registry Service that manages items described semantically and can perform semantic enrichment to better enable search and discovery for information.

The focus of Semantic Mediation Service for this testbed was to focus on transformation of XML document expressed in XML schema using transformation based on XSLT. One of the requirement was to define a ebRIM profile to represent Schema and Schema Mapping served by aebRIM CSW implementation and integrate it with the Semantic Registry. A review of the existing standards to represent schemas and schema mappings in a registry was conducted. Only DCAT and ADMS were found relevant but no profile were defined to accommodate the specificities of schemas and schema mappings such as source and target schema in a mapping. A review of the ebRIM model in CSW was also performed and concluded that an extension was needed to represent schema and schema mapping information in the ebRIM model.

Instead of defining a separate service that manages the schema and schema mapping separately, we decided to extend the core Semantic Registry Information Model by creating a profile to represent them semantically. The benefit of this approach is to test how well the core model can be extended and how well the Semantic Registry REST API can be reused to accomodate different domain models and convenience service APIs. We decided to implement the Semantic Mediation Service as convenience service on top of the Semantic Registy that performs validation and transformation between schemas (including finding a chain of transformation between two schemas). The Semantic Mediation Service would delegate the CRUD operations for schemas, schemas mappings to entirely to the Semantic Registry. We also decided to use a REST API that provide hypermedia controls with JSON-LD representation to lower the bar of integration with web clients, as well as getting Linked Data representation (RDF/XML, Turtle) to describe schemas and schema mappings for machine to machine support.

The focus of Semantic Portrayal Service for this testbed was on storing and accessing portrayal information managed by the semantic registry to support symbology mediation and rendering. Instead of defining a separate service that manages the portrayal information separately, we decided to extend the core Semantic Registry Information Model by creating a profile to represent this information semantically. The benefit of this approach is to test how well the core model can be extended and how well the Semantic Registry REST API can be reused to accommodate different domain models. We decided to implement the Semantic Portrayal Service as convenience service on top of the Semantic Registry that performs portrayal information search and rendering. The Semantic Portrayal Service would delegate the Creation/Update/Delete operations for portrayal information entirely to the Semantic Registry. We also decided to use a REST API that provide hypermedia controls with JSON-LD representation to lower the bar of integration with web clients, as well as getting Linked Data representation (RDF/XML, Turtle) to describe portrayal information for machine to machine support.

To facilitate the integration of clients with the Semantic Mediation Service, we recommended the use of REST API supporting the encoding of the SRIM Schema Application profile in Linked Data format using RDF/XML, Turtle, N-Triples and JSON-LD. We also recommended to accomodate Level 2 (Resources with HTTP Verbs) and Level 3 (Hypermedia-driven) of the Richardson Maturity Model. We choose the Level 3 Hypermedia-driven API using the Hypermedia Application Language (HAL+JSON), which is gaining in popularity in the REST community. To favor reusability of functionalities, all the CRUD operations of the schemas and schema mappings were implemented by the Semantic Registry. The Semantic Mediation Service was build a convenience service on top of the Semantic Registry to provide search capabilities, validation, transformation path calculation and actual transformation of document based on the path calculated from the schema mappings managed by the registry.

To align better with current rendering engine implementation and current descriptive standard for Portrayal (SE, SLD), we decided to align the portrayal ontology closer to the OGC Symbol Encoding (SE)and SVG. We developed the Graphics and Symbolizer ontologies that are closely with these standards, but provide mechanism to support future extensions for more complex stlying scenarios.

The Semantic Portrayal REST Service delegates the CRUD operations on portrayal information to the Semantic Registry which implements the SRIM Portrayal Profile. The Semantic Portrayal Service implements a REST API Level 2 and Level 3 on Richardson Maturity Model. We implemented the Level 3 Hypermedia-driven API using the Hypermedia Application Language (HAL+JSON), which is gaining in popularity in the REST community. The Semantic Portrayal Service should be a convenience service build on top of the Semantic Registry containing Portrayal information by providing search capabilities and rendering points for rendering symbol glyphs for legend and map rendering of geospatial data.

DCAT is an RDF vocabulary designed to facilitate interoperability between data catalogs published on the Web. By using DCAT to describe datasets in data catalogs, publishers increase discoverability and enable their applications to easily consume metadata from multiple catalogs. It further enables decentralized publishing of catalogs and facilitates federated dataset search across sites. Aggregated DCAT metadata can serve as a manifest file to facilitate digital preservation.

The DCAT Application profile for data portals in Europe (DCAT-AP) is a specification based on the Data Catalogue vocabulary (DCAT) for describing public sector datasets in Europe. Its basic use case is to enable cross-data portal search for data sets and to allow public sector data to be easily searchable across borders and sectors. This can be achieved by the exchange of descriptions of datasets among data portals.

In February 2015, the ISA² programme of the European Commission has started an activity to revise the DCAT-AP, based on experience gained since its development in 2013. The outcome of this effort was the publication of DCAT-AP 1.1.

The European Data Portal is implementing the DCAT-AP as the common vocabulary for harmonizing descriptions of over 258,000 datasets harvested from 67 data portals from 34 countries. The DCAT-AP is used in the Open Data Support service initiated by the European Commission with the purpose of realizing the vision of European data portals.

GeoDCAT-AP is defined as an extension of DCAT-AP for describing geospatial datasets, dataset series, and services. It provides an RDF syntax binding for the union of metadata elements defined in the core profile of ISO 19115:2003 and those defined in the framework of the INSPIRE Directive. Its basic use case is to make spatial datasets, data series, and services searchable on general data portals, thereby making geospatial information better searchable across borders and sectors. This can be achieved by the exchange of descriptions of datasets among data portals.

ADMS is a profile of DCAT that is used to describe semantic assets (or just 'Assets'). These assets are defined as highly reusable metadata (e.g. xml schemata, generic data models) and reference data (e.g. code lists, taxonomies, dictionaries, vocabularies) that are used for eGovernment system development.

The ADMS model is intended to facilitate federation and co-operation. Like DCAT, ADMS has the concepts of a repository (catalog), assets within the repository that are often conceptual in nature, and accessible realizations of those assets, known as distributions. An asset may have zero or multiple distributions. As an example, a W3C namespace document can be considered to be a Semantic Asset that is typically available in multiple distributions, one or more machine processable versions and one in HTML for human consumption. An asset without any distributions is effectively a concept with no tangible realization, such as a planned output of a working group that has not yet been drafted.

ADMS is an RDF vocabulary with an RDF schema available at its namespace http://www.w3.org/ns/adms . The original ADMS specification published by the European Commission [ADMS1] includes an XML schema that also defines all of the controlled vocabularies and cardinality constraints associated with the original document.

Project Open Data provides the implementation guide and associated resources for the Federal Executive Order on open data and data management, M-13-13 “Managing Information as an Asset,” which includes the standardized metadata schema that all CFO Act agencies are required to use to publish their enterprise data inventories.

The Project Open Data Metadata Schema is a JSON-based implementation of the W3C DCAT vocabulary. This standard is currently implemented by multiple data catalog platforms as well as state and local governments.

Typically, POD documents are often published in Web Accessible Folder (WOF) and harvested by catalogs such as data.gov. The intent of POD is to lower the bar of complexity neededto represent data information by providing guidelines and recommended metadata. This enables a better search and discovery for datasets within the US goverment.

The standard ISO 19115 defines the schema required for describing geographic information and services that is encoded in XML format. It provides information about the identification, the extent, the quality, the spatial and temporal aspects, the content, the spatial reference, the portrayal, distribution, and other properties of digital geographic data and services. The standard ISO 19115 is applicable to:

ISO 19115-1 defines:

Though ISO 19115-1 is applicable to digital data and services, its principles can be extended to many other types of resources such as maps, charts, and textual documents as well as non-geographic data. Certain conditional metadata elements might not apply to these other forms of data.

ISO 19139 defines the XML-based implementation for ISO 19115. ISO 19115-1:2014 [ISO19115-1] has superseded ISO 19115:2003. At the date of publication of this document, the XML-based implementation of ISO 19115-1:2014 (namely, ISO 19115-3), was finalised but not yet officially released.

This International Standard specifies the procedures to establish, maintain, and publish registers of unique, unambiguous and permanent identifiers and meanings that are define items of geographic information. In order to accomplish this purpose, the standard specifies elements of information that are necessary to provide identification and meaning to the registered items and to manage the registration of these items.

SHACL is an RDF vocabulary for describing RDF graph structures. These graph structures are captured as "shapes", which correspond to nodes in RDF graphs. These shapes identify predicates and their associated cardinalities, and datatypes. Additional constraints can be associated with shapes using SPARQL or other languages which complement SHACL. SHACL shapes can be used to communicate data structures associated with a process or interface, to generate or validate data, or to drive user interfaces.

Most applications that share data do so using prescribed data structures. While RDFS and OWL enable one to make logical assertions about the objects in some domain, SHACL (Shapes Constraint Language) describes data structures. Features of SHACL include:

One of the primary functions of the Semantic Registry is to support search and discovery on a large variety of items using a unified API. The Semantic Registry was tested to handle different item types including Datasets, Services, Schemas, Schema Mappings, as well as Portrayal information such as Symbols, and Portrayal Rules, and Feature Type Styles. To integrate the different encoding standards of this information, including ISO 19139, NMIS, ebRIM Schema Profile, and DCAT, a number of semantic mappers were implemented. These semantic mappers link each standard to the adequate SRIM profiles, and are used by harvesters to extract information from various sources of information.

We found out that a Linked Data encoding (DCAT) of information is easier to integrate than XML encoding because the latter requires code to explicily define the mapping between the syntactic and semantic encoding. The XML encoding of information based on XML schema tends to be more unforgiving when validating data. Another advantage of using a Linked Data approach is that it favors reusability of information that can be created and managed in a decentralized way using a common encoding framework.

One of the biggest challenges when importing data into the system is the validation of the data. The RDF model provides a powerful framework to express any property of a resource by using vocabularies from different ontologies, and it can accomodate easily to partial/incomplete information. However, this flexibility causes difficulties when attempting to validate the data. Due to the limited time for this testbed, we decided to postpone the exploration of SHACL to address this issue for the next testbed. SHACL can provide a powerful way to validate data, define the shape of graph to be processed by the service.

This section summarizes the list of issues found when mapping ISO 19139 to a semantic representation with data coming from a variety of CSW sources (including data.gov and Geoplatform.gov). Some of these issues come from malformed metadata and ambiguities in the ISO 19115 standards, while others come from a lack of policies from agencies that publish metadata. These issues impede interoperability and integration of information in addition to search and discovery. The usage of Linked Data instead of XML encoding will address many of these problems, but not the ones related to policies.

The Semantic Registry Service was designed to manage multiple registers that are capable of containing item classes from different application profiles. To support the testbed 12, we implemented three different registers:

These registers were populated by a harvester service which is integrated with the Semantic Registry Service and accessible by a hypermedia-driven REST API. The harvester service was designed to be extensible and to support multiple types of data sources, including documents extracted from a resolvable URL (Project Open Data, DCAT , ISO 19139, FDGC CSGDM documents), and advanced web services such as CSW, CKAN, Web Accessible Folder, and ESRI Web services. These plugins called harvester types describe the list of parameter descriptors needed by the harvester. An instance of a harvester type is called a harvester source and provides binding of the parameters to values. A harvester source can be triggered for harvesting manually or a given schedule, and the harvester results are returned with statistics (number of harvested objects successfully imported, number of failures) as well as the list of item identifiers. Due to limited time for implementation, only synchronous calls to harvesters are supported. Future development will handle asynchronous harvesting with on demand status reports.

The items managed by the service are stored in a NoSQL store, and are indexed and managed in a RDF store to support graph analytics and SPARQL queries.

The initial objective of the testbed was to provide a DCAT REST API, which focused on the search and discovery of dcat:Datasets. However, promoting the DCAT model to the superset SRIM model also necessitated a promotion of the REST API to manage registers and harvester types and sources, and to handle more general items, including Portrayal items, Schema and Schema Mapping items.

A review of existing implementations that use DCAT datasets showed that the only consensus in how to access the information through a REST API, was the use of a SPARQL query protocol. Using an OGC filter was not considered adequate enough for complex queries of RDF data, as SPARQL provided a more compact and standardized way to query linked data. One of the main considerations when designing the REST API for the Semantic Registry was to make it accessible for web clients, which primarily operate in JSON, and to bridge the gap between linked data and JSON, the Semantic Registry uses the W3C JSON-LD. The use of JSON-LD context allows the conversion of RDF models to JSON representations and vice versa. Another objective of the API was to provide a degree of separation between the server and client implementation, to allow the API to evolve in the future without breaking client ecosystems. To achieve this, the Semantic Registry uses Hypermedia Links which provide a powerful mechanism to decouple clients and servers. This corresponds to the Level 3 REST API on the Richardson Maturity Model.

To implement a Level 3 REST API, we adopted the IETF standard candidate Hypermedia Application Language (HAL), a popular standard candidate which is widely used by JSON hypermedia REST APIs.

We also acknowledge that many web frameworks (such as AngularJS) are designed for Level 2 APIs and construct URLs on the client side to access the different states of a web application. To accomodate these frameworks, we decided to also implement a Level 2 REST API by providing well-defined URL patterns to access the artifacts of the service (registers, items,harvesters types, harvester sources) and a unique identifier for each artifact. The responses of Level 2 are identifical to those of Level 3, except for the exclusion of the hypermedia links to other states. The REST API endpoints URL pattern documented in Appendix D are considered informative only not normative.

In addition to the Level 2 and Level 3 REST APIs that will mostly be used by web clients, we added support for Linked Data API that will mainly be used by machines. Each REST endpoint of the Semantic Registry Service also supports a Linked Data output in RDF/XML, Turtle and N-Triples formats.

Furthermore, each Register endpoint also provides a GeoSPARQL endpoint that permits advanced SPARQL queries on the Linked Data representation of the items managed by each register.

To evaluate interoperability aspects in multi-catalog type environments, the testbed considered a number of solutions. Each solution involved various types of catalogue services, for example, CSW featuring ISO based metadata and OpenSearch, other CSW offering a SOAP binding, and support for DCAT using RDF.

Several architectural solutions could be used to establish a multi-catalogue environment, and four key architectural solutions were identified by the Testbed. The identified solutions differ in a variety of ways, including the entry point for client applications and the computational balance between the client application and the services.

The first solution for a multi-catalogue environment includes a client application that can query the various catalogue services directly. This requires the client application to prepare appropriate queries for each catalogue service and to collate the search results when they are returned by the services.

The second solution involves the selection of one of the catalogue services to initiate a distributed search. In this case, the client application only needs to prepare queries to send to the cascading catalogue service. Upon receiving a request from the client, the cascading catalogue service then adapts the request to forward to other catalogue services and returns responses from the other services, as well as results from its own catalogue.

The third solution involves the harvesting of metadata from one or more source catalogue services into a single target catalogue service. Harvesting is ideally conducted at a scheduled time and not when a query is received from the client. The client application can then query the target catalogue service to discover resources published by both the source and target catalogue services.

The fourth solution involves the replication of metadata between a federation of catalogue services. Replication would ideally be conducted at a scheduled time and not when a query is received from the client. The client application can then query any catalogue service to discover resources published by any catalogue service.

For this tesbed, the integration with the OGC Catalog Services was accomplished by using a Harvester Service.

A number of clients were successfully integrated with the Semantic Registry, as illustrated by the following figure: