OGC Testbed-15: Semantic Web Link Builder and Triple Generator

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

Efforts in OGC Testbeds-11 [1] and 12 [2, 3] targeted semantic mediation support. The focus was exploring the high-level description of ontologies and the metadata needed to enable search on controlled vocabularies. As part of the Testbed-12 activity, a REST API was developed to access vocabulary metadata. This Semantic Registry Service was used as an aide in a first pass of the data in the further enrichment of an existing knowledge base.

The work described in this ER utilizes the vocabulary described in the Semantic Registry Information Model (SRIM) as an enabler for further knowledge base population. The SRIM is defined as a superset of the W3C DCAT standard and encoded as an OWL (Web Ontology Language) ontology. However, there should be restrictions in place (such as mandatory, recommended or optional fields) that cannot be captured with OWL, and require human interpretation. The Shape Constraint Language (SHACL) [4, 5] is a W3C standard that provides a framework to define the shape of the graph data. Proposed future work included investigation of SHACL shapes for its applicability to application profiles, form generation and data entry, data validation and quality control of linked data information.

The work described in this ER was focused on enriching geospatial structured data sources. The tools created in this Testbed harvest semantic data sources and attempt to correlate and fuse this information. The hydrological features for the Richelieu River/Watershed were the data source for this use case. One of the challenges in the work was to overcome mapping the ontology vocabulary in the Semantic Registry Information model to other ontologies used in geospatial databases. The alignment and "mapability" needs to be addressed to confirm the relation and/or add more information to the relation.

The high-level goal of the work described in this ER was to perform knowledge base population on the hydrological entities in the Richelieu River/Watershed. The ultimate goal was to see if the solution for this use case could be generalized for multiple problems.

This section describes the current challenges related to data integration and fusion.

Before we delve into how to implement a robust integration system, it is important that we understand the different types of data that are available on the network. Data can be placed into three categories: unstructured, semi-structured, structured.

While this categorization of data is well understood and widely adopted in the industry, it misses an important aspect: whether the data can be understood by machine-based algorithms or not. Understanding data means that computers can unambiguously interpret the meaning of information and be capable of inferring new information from data. This capability is the realm of Linked Data and the Semantic Web. Semantics are required for data and service interoperability. Semantics are also imperative for machine-to-machine understanding, reasoning (inference), and automation. Semantics greatly aid in search and navigation. They constitute the unifying means for interrelating heterogeneous data. They also aid in data abstraction, categorization, organization, and validation. Finally, semantics give data context and unambiguous meaning.

Many people use the term “data format” to suggest that they have a common, interoperable data model. In the world of formal data modeling, this is considered to be flawed thinking that is fraught with system interoperability challenges. This thinking tends to surface in rushed stovepipe development efforts, and in settings where data modeling experts are absent. (System engineers and software developers who lack data modeling expertise notoriously skirt formal data models, and often misinterpret or vary from data standards.) For example, many developers make the simple mistake of treating a file format as their data model for interoperability. First of all, a (file) format is simply a convenient encoding for point-to-point data exchange purposes. A format is not a logical data model, per se, although it may have a formal logical data model behind it. Whereas a well-designed file format like Shapefile or KML may resolve schema and syntax issues, they do not explicitly deal with the semantics and context of the content it exchanges. Likewise, a collection of (file) formats is not a data model. File formats may actually hinder interoperability because each format requires custom mapping and translation software to another format, and the transformed results often do not align with sound, industry-approved coherent data models that were designed with interoperability in mind (as well exemplified by the work at OGC). Whereas file formats may be effective convenience mechanisms for point-to-point data exchanges, they are not good interoperability mechanisms for open service-based platforms and environments with many-to-many data exchange nodes. In summary, they tend to keep us from achieving the desired higher level of uniformity, logical consistency, and semantic harmony we seek for a System-of-Systems.

In a famous article of Scientific American [10] Tim Berners-Lee described the aim of the Semantic Web bringing the web to its full potential. To enable wider adoption of the Semantic Web, the term of Linked Data was introduced in 2008 [11], which provides a simplified view of semantic web as a web of linkage between data nodes. The idea of linked data is similar to the web of hypertext, but the semantic web is not merely about publishing data on the web, but more about making links in such a way that a person or a machine can explore the data. The linked data leads to other related data. The semantic web is also constructed in such a way that it can be parsed and reasoned about. The web of hypertext is constructed with links anchored in HTML documents, but the semantic web is constructed in such a way that arbitrary links between entities are described by Triples in RDF. URIs are used to identify any kind of concept. [12]

The following describes some of the important concepts of the semantic web:

The rise of the semantic web and linked open data has helped create a large number of available open data sources that could be explored. They could contain controlled vocabularies that structure and constrain the interpretation of the available data. Vocabularies can for example be defined as ontologies using RDF Schema and OWL, linked data with SPARQL, or as constraints that can be defined using SHACL. Ontologies define the meaning of the linked data, which can be modelled as a directed labelled graph. The nodes represent resources and the edges represent properties, assigned to the meaning in the ontology. Access to vocabularies is often provided at a SPARQL endpoint. SPARQL and its geospatial extension GeoSPARQL are now well-established standards for querying Linked Data representation that contains geospatial information.

SPARQL: SPARQL Protocol and RDF Query Language, the generic RDF query language.

Unfortunately, in practice a lot of resources and properties do not have resolvable resource URIs, so the exploration of other linked data is hindered. The principles of 5-star linked open data, which are described in the next subsection, aim to resolve some of the hindrance. Figure 1 illustrates the 5-star principles.

As a means to encourage individual users and governments to implement and improve on linked data [12] in 2010 Tim Berners-Lee developed a 5-star system for grading linked open data. Linked data is not necessarily linked open data, where the data is published under an open license. Linked data can also be 5-star (see Figure 1) within an internal system, but linked open data is data that follows the following graded scale:

Semantic mediation is defined as the transformation from one conceptual model to another, in particular from one ontology to another. Instances of the target classes are created from the values of instances of the source classes. Related work on this topic was documented in OGC Testbed-11 Symbology Mediation ER (OGC 15-058 [1]).

Semantic Mediation was addressed to some extent in OWS-8 [13], OWS-9 [14] and OWS-10 [15]. Those Testbeds were mostly focused on performing semantic mediation for taxonomies. For example, gazetteers such as the Geographic Names Information System (GNIS), GEOnet Names Server, and Geonames often use different taxonomies for classifying feature types. To support semantic mediation, mappings are required from one concept in a source taxonomy to another one in the target taxonomy (using SKOS mapping relationships such as skos:exactMatch, skos:broadMatch, skos:closeMatch). The semantic mediation was demonstrated using the OGC Web Feature Service-Gazetteer (WFS-G), however the mediation was performed as black box on syntactic representation of the features (using Geography Markup Language (GML)). In OWS-10, the hydrology sub-thread of CCI [15] attempted to address a more general approach for mediation by defining some mapping between two different hydrologic models. However, the solution was based on UML tools that perform the mapping and no formal model was defined to encode the semantic mapping. The mapping between different hydrological models was formalized through mediation by means of shared semantics. A two-step mapping approach was applied using the HY_Features model as mediating model. In the first step, the hydrological models is mapped to common feature concepts of HY_Features. In the second step, the HY_Features model is “re-mapped” to the target models. To achieve semantic interoperability among the National Hydrography Dataset Plus (NHD)+ - and National Hydrographic Network (NHN) data models, two separate mappings are required: (1) mapping of NHD+ features to the equivalent concepts of HY_Features, and (2) mapping NHN features to HY_Features equivalents.

To address the semantic mediation of symbology, Testbed-11 addressed semantic mediation by representing information as linked data (see OGC 15-058). The goal of Testbed-11 was to formally address the semantic mediation for taxonomies by defining extensions to GeoSPARQL (geosparql:skosMatch) to perform the semantic mapping and providing an extensible, sharable encoding of the semantic mappings that can be processed by machine. A RESTful Semantic Mediation Service was demonstrated to perform semantic mapping between the Homeland Security Working Group (HSWG) Incident Model to the EMS Incident Model. For this purpose, existing linked data standards (RDF, OWL, SPARQL) were leveraged to represent semantic mappings. These semantic mappings can be managed by a semantic mediation service to perform transformation between two models. OGC 15-058 [1], Symbology Mediation Engineering Report from Testbed-11 describes the basic principles of semantic mediation using the example of two portrayal ontologies that need to be aligned in an ad hoc manner. In Testbed-12 (OGC 16-059 [2]),the semantic mediation work in Testbed-12 was closely related to the semantic portrayal work described above and built on the achievements from Testbed-11. Testbed-12 focused on the usage of a schema registry to store information about schemas and schema mappings to support ad hoc transformations between source and a target schemas. Schema mappings were considered as a simple form of semantic mediation, but were defined without explicit formalization of the underlying semantic knowledge required to map from one schema to another. For that reason, the idea was to design a Semantic Mediation Service REST API and integrate the API with the Semantic Registry and the CSW ebRIM profile for Schema Registry.

Since the conclusion of Testbed-13, W3C has standardized the SHACL specification [5]. The SHACL specification shows a lot of overlap with the work performed during Testbed-11 (see OGC 15-058). In particular, SHACL defines Shapes, Parameter, Function and Rules that have been defined in the semantic mapping ontology from Testbed-11. A comparison of SHACL with OWL was performed during Testbed-14 (OGC 18-094r1) and a model was defined to represent application profile for linked data. Within this context, an application profile defines a subset of concepts and properties from one or more ontologies to support mediation and in many use cases, semantic mediation is needed between two application profiles. W3C is currently conducting an effort to define application profiles for Linked data [16] and many application profiles such as DCAT-AP and GeoDCAT are encoded using the SHACL standard. One of the main efforts of Testbed-15 was to update the mapping ontology by using the SHACL standard. By doing so, the semantic mediation can be considerably simplified and be usable with emerging SHACL APIs. This would favor interoperability and sharing of semantic alignments.

When integrating multiple linked data sources, equivalent entities may be identified with different identifiers. The goal of the deduplicaton phase is to identify entities that are similar semantically in order for them to be merged into one single entity in the next phase: semantic fusion. To perform this task, the rules of linkage between two entities that are similar need to be defined.

The literature suggests a number of Linked Data frameworks addressing this task (also called Linked Discovery). Silk - Link Discovery Framework [17] uses the declarative Silk - Link Specification Language (Silk-LSL) defined in XML and a proprietary path syntax to define the linkage rules. Data publishers can specify which types of RDF links should be discovered between data sources as well as which conditions data items must fulfill in order to be interlinked. These link conditions can apply different similarity metrics to multiple properties of an entity or related entities which are addressed using a path-based selector language. The resulting similarity scores can be weighted and combined using various similarity aggregation functions.

Silk accesses data sources via the SPARQL protocol and can thus be used to discover links between local and remote data sources. SILK uses a multi-dimensional blocking technique (MultiBlock [18]) to optimize the linking runtime through a rough index pre-matching. To parallelize the linking process, SILK relies on MapReduce. The framework allows user-specified link types between resources as well as owl:sameAs links. SILK incorporates element-level matchers on selected properties using string, numeric, temporal and geo-spatial similarity measures. SILK also supports multiple matchers, as it allows the comparison of different properties between resources that are combined together using match rules. SILK also implements supervised and active learning methods for identifying Link Specifications (LS) for linking. One of the shortcomings of the framework is that it uses a proprietary document-based format for configuring linkset specification and custom path syntax. The model does not provide a mechanism for defining additional constraints on the nodes referred by the paths. The SILK framework comes with a workbench which allows to define the link set specification in a visual way (see Figure 2).

Another, and more recent framework, is the Linked discovery framework for MEtric Spaces (LIMES). This framework implements time-efficient approaches for large-scale link discovery based on the characteristics of metric spaces. [19] It is easily configurable via a configuration file as well as through a graphical user interface. According to its web site: [20]

"LIMES implements novel time-efficient approaches for link discovery in metric spaces. Our approaches facilitate different approximation techniques to compute estimates of the similarity between instances. These estimates are then used to filter out a large amount of those instance pairs that do not suffice the mapping conditions. By these means, LIMES can reduce the number of comparisons needed during the mapping process by several orders of magnitude. The approaches implemented in LIMES include the original LIMES algorithm for edit distances, HR3 [21], HYpersphere aPPrOximation algorithm (HYPPO) [22], and ORCHID [23]. Additionally, LIMES supports the first planning technique for link discovery HELIOS [24], that minimizes the overall execution of a link specification, without any loss of completeness. Moreover, LIMES implements supervised and unsupervised machine-learning algorithms for finding accurate link specifications. The algorithms implemented here include the supervised, active and unsupervised versions of EAGLE [25] and WOMBAT [26].

The LIMES framework consists of eight main modules of which each can be extended to accommodate new or improved functionality. The central modules of LIMES is the controller module, which coordinates the matching process. The matching process is carried out as follows: First, the controller calls the configuration module, which reads the configuration file and extracts all the information necessary to carry out the comparison of instances, including the URL of the SPARQL-endpoints of the knowledge bases S (source) and T(target), the restrictions on the instances to map (e.g., their type), the expression of the metric to be used and the threshold to be used.

Given that the configuration file is valid w.r.t. the LIMES Specification Language (LSL), the query module is called. This module uses the configuration for the target and source knowledge bases to retrieve instances and properties from the SPARQL-endpoints of the source and target knowledge bases that adhere to the restrictions specified in the configuration file. The query module writes its output into a file by invoking the cache module. Once all instances have been stored in the cache, the controller chooses between performing Link Discovery or Machine Learning. For Link Discovery, LIMES will re-write, plan and execute the Link Specification (LS) included in the configuration file, by calling the rewriter, planner and engine modules resp. The main goal of Linked Discovery is to identify the set of links (mapping) that satisfy the conditions opposed by the input LS. For Machine Learning, LIMES calls the machine learning algorithm included in the configuration file, to identify an appropriate LS to link S and T. Then it proceeds in executing the LS. For both tasks, the mapping will be stored in the output file chosen by the user in the configuration file. The results are finally stored into an RDF or an XML file. ""

Like the SILK framework, the configuration of LIMES is defined with a proprietary XML format and syntax for expressing rules as illustrated in Figure 3

The Testbed-15 approach to model correlation is based on Open Linked Data Standards (RDF, SHACL, OWL, SPARQL). The rationale is to be able to favor reusability of the correlation rules between two linked data sets, by leveraging the linking properties that is built-in in the linked data framework by referring to URI identifier. Three ontologies were formalized to model metrics Metrics Ontology, similarity Similarity Ontology and correlation Correlation Ontology to support correlation task using a correlation engine implemented for this testbed.

Due to the decentralized nature of the web, different communities can represent the same-real world objects in different way. This results in conflicts. We can distinguish three type of conflict [27].

The first two kinds of conflict are resolved during the semantic mediation and correlation phase of data integration. The third kind of conflict, data conflicts, remain and are not resolved until data fusion and are caused by the remaining multiple representations of same real-world objects.

We distinguish two kinds of data conflict: (a) uncertainty about the attribute value, caused by missing information; and (b) contradictions, caused by different attribute values [27]:

We can distinguish three categories of conflict strategies:

Bleiholder et al (2005) [29] formalize some possible conflict handling strategies summarized in Table 1

The fusion result to an integrated information system should have the following characteristics:

The following table summarizes a number conflict resolution functions found in the literature [29].

This list of functions could be used as a starting point to design an ontology for fusion which defines conflict resolution function (may be based on SHACL functions). Unfortunately, due to time constraints, this would have to be investigated in future Testbeds.

There is a fundamental difference in the interpretation of OWL restrictions versus SHACL constraints. OWL is designed for inferencing, meaning that the interpretation of restrictions leads to OWL making assumptions and inferences about the data. OWL is based on an Open-World Assumption [4], where absent statements and properties can be filled later, and absence of data does not invalidate the statement. The application should infer the missing data. Violations in for example cardinality mean that the OWL processor will assume the violating values must represent the same entity with different URIs. This is because OWL makes a distinction between URI and real-world entity where multiple URIs can represent the same entity. This is also referred to as the Unique-Name Assumption. [4]

SHACL is designed based on a Closed-World Assumption [4], where lack of data invalidates the statement, meaning that the interpretation of restrictions is based on the assumption that the knowledge base is complete or can be assumed to be as complete as possible with the current information. Aside from the built-in constraints, SHACL constraints can be expressed using SPARQL or in JavaScript, making it highly flexible and extensible. In contrast to OWL, where limited data validation is done via inferencing, SHACL separates checking data validity from reasoning and inferring new facts. OWL’s built-in nature of the Open World Assumption and the Unique Name Assumption contradicts established approaches from schema languages and makes the meaning of certain statements (e.g., cardinality) different from what most modelers expect. [4]

Due to the differences in design philosophy and implementation, one of the main advantages of SHACL over OWL is that SHACL is extensible while OWL is limited to the features as defined by the OWL committee. In the end, the SHACL vocabulary is complementary to OWL, but has a higher usability. The usability increase can be experienced through the SHACL shapes in the definition of constraints and constraint targets, as well as the built-in constraint types.

When the SHACL standard was released by the W3C in 2017, it introduced the concept of shapes. SHACL shapes are described in terms of RDF graphs, which is then referred to as a shapes graph, and follow a hierarchy of shapes based on the RDF Schema language. SHACL aims to validate RDF graphs against the shapes, where the data being validated is called a data graph.

There are two types of shape: node shape that declare constraints directly on a node and property shape that declare constraints on the values associated with a node through a path. Node shapes declare constraints directly on a node e.g., node kind (Internationalized Resource Identifier (IRI), literal or blank), IRI regex, etc. Property shapes declare constraints on the values associated with a node through a path, e.g., constraints about a certain ongoing or incoming property of a focus node; cardinality, datatype, numeric min/max, etc. (see Figure 4)

SHACL shapes may define several target declarations. Target declarations specify the set of nodes that will be validated against a shape, e.g., directly pointing to a node, or all nodes that are subjects to a certain predicate, etc. The target declarations of a shape in a shapes graph are triples with the shape as the subject and one of sh:targetNode, sh:targetClass, sh:targetObjectsOf or sh:targetSubjectsOf as a predicate.

Shapes also declare constraints which are constraints on the focus nodes and value nodes of their properties. Constraints to be applied to the target, e.g. cardinalities, ranges of values, data types, property pairs, etc. They can also declare rules that can be used to add inferences or perform mapping from one model to another (see Figure 5 ).

A more detailed description of the SHACL model is shown in Figure 6. The SHACL specification has a number of extensions that enable expanding the expressiveness of the validation. The SHACL-SPARQL extension provides mechanism to add Constraint Component plugins and Rules based on the standard SPARQL. The SHACL advanced features provides additional mechanism to add rules for inferencing and transformation. SHACL-JS provides JavaScript extensions for SHACL, which could implement functions for constraint component validators, target selections and rules.

RDF terms produced by targets are not required to exist as nodes in the data graph. Targets of a shape are ignored whenever a focus node is provided directly as input to the validation process for that shape. A focus node is an RDF term that is validated against a shape using the triples from a data graph.

An RDF graph uses namespaces to anchor to the appropriate (ontology) vocabularies, which can be stored in industry standard formats like Turtle, JSON-LD and RDF/XML. SHACL itself is defined as part of a namespace as well. There are a number of standard namespace prefixes that can be encountered as part of the shapes definition:

While SHACL is defined using OWL, there was no UML diagrams available for the specification. To support the implementation of the SHACL engine, a UML Object Model compliant with the SHACL ontology has been defined as part of this work. The model may not capture all the nuances of the specification. However, the model was designed to have share a common understanding. The model should not be considered normative, but only informative. The model provides extensions points for Functions, Rules, Constraint Components and TargetTypes. The built-in and most commonly used constraint components can be directly set to the shapes to simplify their encodings. An overview of the model is shown in Figure 7.

The next subsections describe the other first-class business objects of the SHACL specification in more detail.

Constraint Components are at the core of the validation process. SHACL defines the concept of constraint components which are associated with shapes to declare constraints. Each node or property shape can be associated with several constraint components.

Constraint components are identified by an IRI and have two types of parameters: mandatory and optional. The association between a shape and a constraint component is made by declaring values for the parameters (called parameter bindings). The parameters are also identified by IRIs and have values. Most of the constraint components in SHACL Core have a single parameter and follow the convention that if the parameter is named sh:p, the corresponding constraint component is named sh:pConstraintComponent. The Constraint Component model is shown in Figure 8.

Constraint components are associated with validators, (see Figure 9), which define the behavior of the constraint. Writing custom Constraint Component is considered as an advanced feature of the system. However, this provides a powerful extension mechanism to represent more advanced constraints such as spatial-temporal constraints.

For example, the DASH namespace includes a collection of SHACL constraint components that extend the Core of SHACL with new constraint types including value types, string-based, property pairs, relationship constraint components.

A profile in the context of linked data is defined as "a named set of constraints on one or more identified base specifications, including the identification of any implementing subclasses of datatypes, semantic interpretations, vocabularies, options and parameters of those base specifications necessary to accomplish a particular function." Application profiles are included in this definition.

NOTE: ISO and OGC define an Application Profile as a set of one or more base standards and - where applicable - the identification of chosen clauses, classes, subsets, options and parameters of those base standards that are necessary for accomplishing a particular function [ISO 19101, ISO 19106].

A profile is based on an existing specification, like a standard, that can either be another profile or another specification. A profile can be expressed using a single specification, or there can be more than one component. A profile, in the context of linked data, may consist of:

Profiles serve to enable different views of the same data, making it possible to define a specification of the data to fit specific user or application needs. The original vocabulary does not change, but the specific representation does. They can take a number of forms and can have a variety of relationships to existing vocabularies, standards, and other profiles. The applicability of the profile can be tested for goodness of fit to assess the appropriateness for a certain application.

In the context of application profiles, a profile may consist of one or more (sub)sets of terms that gain information from one or more other vocabularies deemed appropriate for a specific application operating in a specific context.

The NRCan datasets contains features classes shown in Figure 13

The NRCan datasets contains features properties shown in Figure 14

The following is a sample of original hydrologic features defined by NRCan.

To perform the correlation tasks, the labels were normalized by removing the typing prefix such as "Water body:" or "Plan d’eau: ". By doing so the matching scoring was given better results against DBPedia and Wikidata. The following shows a sample of cleaned feature labels:

As the above data sources do not provide any specialized information on wells and stations, there were few useful properties beyond the label and the type of the feature that could be leveraged for performing the fusion against the open source data. Only water bodies and catchments provided some results.

The most successful and popular wiki by far is probably Wikipedia, the largest online encyclopedia created and maintained by a global distributed author community. Wikipedia is translated into 306 languages and the English version contains more than 5.9 million articles. However, Wikipedia is intended for human reading and cannot be processed by machines. To address this challenge, semantic web technology was used to make better use of the knowledge embedded in Wikipedia pages. DBPedia is a community effort to extract structured information from Wikipedia and to make this information available on the web. DBPedia allows the user to ask sophisticated queries against Wikipedia (using SPARQL), and to link other data sets on the web to Wikipedia data.

This section discusses the different data sources that are used as data input in the implementation, and the different challenges that they present.

We also identified the following relevant properties from DBPedia:

Wikidata is a project of Wikimedia Deutschland which started on October 30, 2012. The aim of the project is to provide data which can be used by any Wikipedia project, including Wikipedia. Wikidata does not only store facts, but also the corresponding sources, so that the validity of facts can be checked. Labels, aliases, and descriptions of entities in Wikidata are provided in more than 350 languages. Wikidata is a community effort, i.e., users collaboratively add and edit information. Also, the schema is maintained and extended based on community agreements.

The relevant properties from Wikidata for this project are shown in Figure 16:

The GeoNames geographical database is available for download free of charge under a creative commons attribution license. The database contains over 25 million geographical names and consists of over 11 million unique features including 4.8 million populated places and 13 million alternate names. All features are categorized into one of nine feature classes and further subcategorized into one of 645 feature codes. There are a number of features code related to Hydrologic features codes:

The feature code H represents hydrofeatures such as stream, lake, bay, channel, pond, etc. Table 5 has some relevant feature types that are relevant for this testbed:

OpenStreetMap is built by a community of mappers who contribute and maintain data about roads, facilities, waterways, trails, cafés, railway stations, and much more, all over the world.

Figure 17 shows relevant features for the NRCan datasets.

Freebase is a Knowledge Graph (KG) announced by Metaweb Technologies, Inc. in 2007 and was acquired by Google Inc. on July 16, 2010. In contrast to DBpedia, Freebase had provided an interface that allowed end-users to contribute to the KG by editing structured data. Besides user-contributed data, Freebase integrated data from Wikipedia, Notable Names Database (NNDB), Fashion Model Directory (FMD), and MusicBrainz. Freebase uses a proprietary graph model for storing also complex statements. On December 16, 2014, the Freebase team announced that Freebase would shutdown its services on June 30, 2015. Wikimedia Deutschland and Google plan to integrate Freebase data into Wikidata in the near future – a tool for that will be developed prior to August 2015 – and to then close the Freebase website at the earliest three months later.

The integration pipeline system needs to address a number of challenges to get a unified integrated view of the information from different data sources [34]:

During this Testbed, the participants evaluated the use of an open source SHACL engine. The reference implementation of SHACL implemented by TopQuadrant was investigated. While the engine provides a complete implementation of SHACL-Core and SHACL Advanced Feature, it does not provide fine grained control of the different SHACL artefacts used by the different ontologies used for this Testbed. The integration with external resources such as SPARQL endpoint was not easily implementable, as the reference implementation requires at this point using graphs stored in memory. For this reason, Image Matters implemented its own SHACL engine that provides fine grained control of the SHACL elements such as functions, rules, property shapes, constraint components. The SHACL API was integrated with the correlation engine and is planned to be used for the mediation and fusion engine in the future. The performance of the new engine SHACL was about 10 times faster than the reference implementation for the test cases defined by W3C.

The ontology is defined by the namespace: http://purl.org/ontology/correlation# and uses the preferred prefix corr. The UML model for this ontology is shown in Figure 19.

The Similarity Ontology defines concepts and properties for describing similarity methods to correlate entities described as Linked Data. The ontology uses the standard W3C SHACL Property Shape and SHACL Path model to describe the path and constraints on source and target node that needs to be correlated. The ontology leverages the Metrics Ontology (described below) to refer to metrics used for comparison. This ontology addresses some of the shortcomings of previous linked discovery framework such as SILK and LIMES by providing a model that is based on open standards (RDF, OWL and SHACL) favoring its reusability using standard Linked Data mechanism. This model can be extended by adding new functions (defined in SHACL) and constraint conditions that need to be satisfied on the node values that needs to be compared. The ontology is also modularized in such a way that it can be reused in different domains and applications.

The ontology is defined by the namespace: http://purl.org/ontology/fusion/similarity# and uses the preferred prefix sim. The UML model is shown in Figure 20

There are a large number of metrics (whether distance or similarity metrics) that can be used to compare entity attributes such as lexical, spatial, temporal, semantic, numeric metrics. However, the participants have not identified any existing ontology that formalize these metrics. Therefore, a new ontology module to represent metrics is introduced in this Testbed. The metrics ontology module was defined separately in order to favor reusability for other applications. The ontology provides the core concepts to define metrics and their types that can be classified using different classification scheme. The aim of the ontology is to provide an extensible framework to introduce new metrics when necessary.

The ontology is defined by the namespace: http://purl.org/ontology/metrics# and uses the preferred prefix metric. The UML model of the ontology is shown in Figure 21.

A correlation engine built on top of the SHACL engine was implemented to handle the Correlation LinkSetSpecification. The engine accepts a LinkedSetSpecification, a source and target model, which could be served through a SPARQL endpoint or loaded in memory and parameters to include generated statements and closure of target nodes. The engine was integrated with a RESTful API, so it could be made accessible by web client.

The semantic mediation ontology is used to transform an entity expressed in a given ontology A to an entity expressed in a target ontology B. A first version of this ontology was defined during Testbed-12 to support the semantic mediation service. A number of supporting constructs needed to be introduced such as Rule and Function expressible in RDF. For this Testbed, an update of the ontology was done to leverage the SHACL standard. By leveraging SHACL shapes to select target entities, functions and rules to model transformation, the model is considerably simplified compared to the version developed during Testbed-12. The new ontology introduce only 3 core classes: alignment (mediation:Alignment), class mapping (mediation:ClassMapping) and property mapping (mediation:PropertyMapping) (see Figure 22).

The ontology is defined by the namespace: http://purl.org/ontology/mediation# and uses the preferred prefix mediation.

A semantic mediation service was implemented during Testbed-12. The service managed schemas and schema mapping, and performed transformation from one schema to another. The service followed REST principles and used JSON-LD and the Hypermedia Application Language (HAL). It is anticipated that a variety of clients may use the Semantic Fusion Service, and as a consequence it is difficult to accommodate the needs of every type of client. The REST API will evolve and be modified as more requirements and features are added to the service. As long as the clients are using the semantics of the link relation types, the hypermedia-driven API provides a robust approach to evolve the API without breaking the client ecosystem. For this reason, the service has adopted a hypermedia-driven RESTful API by default, which meets level 3 of the Richardson Maturity Model.

For this Testbed, a REST-based Fusion Service API was implemented with a POST endpoint to perform correlation between two datasets expressed in different ontologies. Future extensions of the service will implement Create-Read-Update-Delete (CRUD) operations to manage LinkSetSpecification, Alignment and Fusion Policies, and to perform mediation and fusion operations. This could be addressed in future Testbeds.

The POST operation for correlation was accessible at the endpoint /correlate. The endpoint accepted a multipart/form-data with the following key-value pairs that are used to configure the correlation engine described above, in Correlation Engine.

The response of the POST operations returns an RDF Model with similarity links and optionally the statements and targets node closure (according the setting of the request parameters).

Figure 23 shows the correlation of the NRCan sample with DBPedia without including the DBPedia triples.

Figure 24 shows the correlation of the NRCAN sample with DBPedia with enriched DBPedia triples.

The REST API for the fusion service can be integrated with a WPS interface. At the time of publishing this ER, the integration was not performed due to lack of time. However, the participants are planning on writing a WPS plugin in GeoServer that will connect to the REST API correlation endpoint of the fusion service. However, the WPS service will not be able to support full CRUD operations. Using the REST API is preferable for making the resource resolvable and to integrate with web clients.

This section discusses web crawling of linked data information. While this capability was not implemented during the Testbed due to time constraints and focus on solving data fusion problem, the participants captured some of the requirements and ideas of what a semantic web crawler would look like in the context of data discovery and data integration. The participants assume that a semantic crawler is configured to harvest linked data information about some entities of interests expressed in a given ontology.

The first problem to overcome is how do you determine what data sources to crawl. The initial step is to find if there a is semantic mapping specification from the entity ontology to other ontologies (see below model about LinkedSet specification). These semantic mappings can be managed in a semantic registry (as demonstrated in Testbed-12 for Semantic Mediation Service) or in a vocabulary management service. Once all the mapped ontologies have been identified, we need to identify data sources that use these ontologies. The search of these data sources (datasets) can be done again using the Semantic Registry which describes metadata about the datasets including the spatial scoping, temporal scoping, ontologies used and access information (SPARQL endpoint for example).

Once the data sources with relevant ontologies have been identified, the crawler will query. For each entity to enrich, a query is formulated to each data source using the associated linkedSet specification to return candidate entities to be correlated and fuse. When an item has an owl:sameAs link to an equivalent entity, the crawler can be configured to follow the link and fetch additional triples from the resolvable URL. The triples are aggregated to be further analyzed and fused. The results of the crawler and fusion process can be stored in an RDF repository or returned through an API in an asynchronous manner.

The implementation of a mediation engine supporting the mediation ontology should be addressed in future work. In particular, the OGC should investigate the transformation from one ontology to another, an application profile to another profile, and be able to transform source query to a target query on the fly by using query rewriting techniques leveraging the semantic mapping specification.

The analysis of conflict resolution needed in the fusion phase was performed in this Testbed. In future Testbeds, the formalization of the different types of conflict and resolution strategies should be formalized. This is so they can be used by a fusion engine to implement the fusion of similar entities coming from multiple sources. The fusion process needs to produce fused entities that are semantically coherent. To be consistent with the other ontologies designed during this Testbed and that favor interoperability and reusability, the ontology should use the SHACL standard. To demonstrate the usage of the fusion ontology, a fusion engine should be implemented to demonstrate the fusion of entities coming out from the correlation phase in the fusion pipeline.

For a future Testbed, a complete fusion pipeline should be demonstrated by leveraging the data to semantic mapping, correlation, mediation and fusion ontologies to perform the integration of entities from multiple sources defined in different ontologies.