OGC Testbed-15: Federated Cloud Provenance ER

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

Provenance provides vital information for evaluating quality and trustworthiness of information on the Web. Therefore semantically interchangeable provenance information must be accessible and there must be an agreement on where and how this information is to be located [1]. The mission statement of the W3C Provenance Incubator Group stated that the provenance of information is critical to making determinations about whether information is trusted, how to integrate diverse information sources, and how to give credit to originators when reusing information. Broadly defined, provenance encompasses the initial sources of information used as well as any entity and process involved in producing a result. In an open and inclusive environment such as the Web, users find information that is often contradictory or questionable. People make trust judgements based on provenance that may or may not be explicitly offered to them.

Provenance is a well-established term in the context of art or digital libraries. In these cases, provenance respectively refers to the documented history of an art object, or the documentation of processes in a digital object’s life cycle [2]. The “e-science community” interest for provenance [3] is also growing, as it is considered as a crucial component of workflow systems [4] that can help scientists ensure reproducibility of their scientific analyses and processes. In law, the concept of provenance refers to the "chain of custody" or the paper trail of evidence. This concept logically extends to the documentation of the history of change of data in a knowledge system [5].

In the context of this study, the term provenance refers to information about entities, activities, and people involved in producing a piece of data or thing, which can be used to form assessments about its quality, reliability, or trustworthiness [6].

Around the year 2006, consensus began to emerge on the benefits of having a community-defined data model and uniform representation for “data provenance, process documentation, data derivation, and data annotation”, as stated in [7].

There are many areas of research and development that have studied relevant aspects of provenance. They can be classified into the following broad categories [8],[4],[7],[9]:

A series of challenges for provenance were launched [7]. The first Provenance Challenge [10] was to test the hypothesis that heterogeneous systems (mostly in the e-science/cyberinfrastructure space), each individually capable of producing provenance data by observing the execution of data-intensive processes, could successfully exchange such provenance observations with each other, without loss of information. The Open Provenance Model (OPM) [7] was proposed as a common data model for the experiment. The second challenge aimed at allowing disparate groups to gain a better understanding of the similarities, differences, core concepts, and common issues across systems. The third challenge aimed at exchanging provenance information encoded with OPM and providing additional profiles. The fourth challenge was to apply OPM to scenarios and demonstrate novel functionality that can only be achieved by the presence of an interoperable solution for provenance. Some of the approaches to address these challenges use Semantic Web technologies [11].

The notion of causal relationships, or dependencies, involving artifacts (e.g., data items), processes, and agents plays a central role in OPM. Using the OPM, one can assert that an artifact A was produced or consumed by a process P, such as “the orthoimage was produced by using a Digital Elevation Model M, and aerial image I and control points C using orthorectification algorithm P.” Here M, I, and C are artifacts, and P is a process. One can also assert a derivation dependency between two artifacts, A1 and A2, without mentioning any mediating process, i.e., “A2 was derived from A1.” Agents, including humans, software systems, etc., can be mentioned in OPM as process controllers. For example, “the orthorectificaton process was controlled by software S managed by agent X. OPM statements attempt to explain the existence of artifacts. Since such statements may reflect an incomplete view of the world, obtained from a specific perspective, the OPM adopts an open world assumption, whereby the statements are interpreted as correct but possibly incomplete knowledge: “A2 was derived from A1” asserts a certain derivation, but does not exclude that other, possibly unknown artifacts, in addition to A1,may have contributed to explaining the existence of A2. Other features of the OPM, including built-in rules for inference of new provenance facts, are described in detail in [10].

In September, 2009, the W3C Provenance Incubator Group was created. The group’s mission was to “provide a state-of-the art understanding and develop a roadmap in the area of provenance for Semantic Web technologies, development, and possible standardization.” The group produced its final report in December 2010 [12]. The report highlighted the importance of provenance for multiple application domains, outlined typical scenarios that would benefit from a rich provenance description, and summarized the state of the art from the literature, as well as in the Web technology available to support tools that exploit a future standard provenance model. As a result, the W3C Provenance Working Group was created in 2011. The group released its final recommendations for PROV in June 2013 [13].

The Core PROV-O standard defines the following core elements (see Figure 1) [13] (see ):

One of the advantages of PROV-O is that it is based on semantic web standards, thus provenance information can be interpreted by machines without ambiguity using well-defined semantics specified by the PROV-O ontology. The model can be extended, in principle, using the standard OWL extension mechanism (subclass, subproperty,and so forth) to address the needs of multiple disciplines. The PROV-O specification defines additional terms that extend the core concepts of the specification. Figure 2 depicts Entities as yellow ovals, Activities as blue rectangles, and Agents as orange pentagons. The domain of prov:atLocation (prov:Activity or prov:Entity or prov:Agent or prov:InstantaneousEvent) is not illustrated.

Most of the provenance storage solutions are currently based on a database controlled by a central authority. This means that if the centralized authority is compromised, the data provenance can also be compromised and be tampered or destroyed. A reliable secure decentralized solution could be more suitable to address this issue.

There are a number of studies that have investigated the role of automated workflows and published best practices to support workflow design, preservation, understandability, and reuse. A summary of the recommendations and their justifications has been summarized by Khan et al [14] by studying workflows from different domains (see Table 1). Their study classifies the recommendations into broad categories related to workflow design, retrospective provenance, the computational environment required/used for an analysis, and better findability and understandability of all shared resources. The recommendations have been informed by a wide corpus of literature [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [15], [25], [26], [27], [28]. Their findings were used to define the CWLProv model, which defines the provenance model for the Common Workflow Language (CWL).

These recommendations can be clustered into broad themes as shown in Figure 3.

While considerable work has been done for provenance of workflow and documents, much less work has been done on securing the provenance information. Secure provenance is of paramount importance to federated cloud computing, yet it is still challenging today. Data provenance needs to be secured because it may contain sensitive/private information. Cloud service providers do not guarantee confidentiality of the data stored in dispersed geographical locations. Unless provenance information is secured and under appropriate access control policies for confidentiality and privacy, the information simply cannot be trusted. [29]. In the past few years, several studies have recognized the importance of securing provenance [29], [30], [31],[32].Lee and al. [31] provide a survey of the provenance security challenges in the cloud.

To guarantee the trustworthiness of data provenance, the data provenance scheme must satisfy the following general data security properties [32]:

In other words, secure trustworthy provenance mechanisms ensure that the integrity of provenance chains are tamper evident, their contents are confidential, and auditors can verify their authenticity without having to know the contents.

Researchers have proposed several data provenance related efforts. The Provenance-Aware Storage System (PASS) was the first scheme to address the collection and maintenance of provenance data at the operation system level [33]. A file provenance system [34] was proposed to collect provenance data by intercepting file system calls below the virtual file system, which requires changes to operating systems. For cloud data provenance, S2Logger [35], was developed as an end to end data tracking tool which provides both file-level and block-level provenance in kernel space. In addition to data provenance techniques and tools, the security of provenance data and user privacy has also been explored. Asghar et al. [32] proposed a secure data provenance solution in the cloud, which adopts a two folder encryption method to improve privacy but at a higher computation cost. In SPROVE [36] provenance data confidentiality and integrity are protected using encryption and digital signature, but does not support provenance data querying capability. The kernel-level logging tool Progger [37] provides log tamper-evidence at the expense of user privacy. There are also efforts which use provenance data for managing cloud environments, such as, discovery of usage patterns for cloud resources, popularized resource reuse and fault management [38].

The freezing and packaging of a runtime environment that includes all the software components and their dependencies used in data analysis workflow is considered today as best practices and has been widely adopted in cloud computing environments where images and snapshots are used and shared by researchers [40]. To distribute system-wide software, various lightweight and container-based virtualization and package managers have emerged such as Docker and Singularity.

Docker [41] is a lightweight container-based virtualization technology that facilitates the automation of application development by archiving software systems and environments to improve portability of the applications on many common platforms including Mac OS X, Linux, Microsoft Windows and cloud instances. One of Docker’s main features is the ability to find, download, deploy and run container images that were created by other developers quickly. Within the context of Docker, the place where images are stored is called a registry, and Docker Inc. offers a public registry called the Docker Hub. You can think of the registry along with the docker client as the equivalent of Node’s NPM, Perl’s CPAN or Ruby’s RubyGems.

Singularity [42] is a cross-platform open source container engine specifically supporting High Performance Computing (HPC) resources. Singularity can import Docker format software images. Singularity enables users to have full control of their environment. Singularity containers can be used to package entire scientific workflows, software and libraries, and even data. This means that you do not have to ask your cluster admin to install anything for you - you can put it in a Singularity container and run.

The sharing and preservation of runtime environment packaging is becoming regular practice in the workflow domain and is supported today by all the leading platforms managing cloud infrastructure and computing services. These cloud providers include Digital Ocean [43], Amazon Elastic Compute Cloud [44], Google Cloud Platform [45] and Microsoft Azure [46]. The instances launched on these platforms can be saved as snapshots to be analyzed or can be recreated to restore the computing state at analysis time.

The Common Workflow Language (CWL) [47] has emerged as a workflow definition standard for the heterogeneous workflow environments. CWL is an open standard for describing analysis workflows and tools in a way that makes them portable and scalable across a variety of software and hardware environments, from workstations to cluster, cloud, and HPC environments. CWL is designed to meet the needs of data-intensive science, such as Bioinformatics, Medical Imaging, Astronomy, High Energy Physics, and Machine Learning. CWL is developed by a multi-vendor working group consisting of organizations and individuals aiming to enable scientists to share data analysis workflows [48]. CWL has been widely adopted by a large number of organizations and been implemented in a large set of open source tools and workflow management systems.

While a common standard for describing analysis workflow is an important step toward interoperability of workflow management systems, it is also important to share and publish results of these workflow executions in an transparent, comprehensive, interoperable and secure manner. This is essential to reduce duplication of workflows, improve the trust and integrity of data and analyses, improve reproducibility of scientific workflows and catalyze the discovery of new knowledge. Currently, there is no common format defined and agreed upon for interoperable and secure provenance, workflow archiving or sharing.

CWLProv [14] has been proposed very recently as a format to represent any workflow-based computational analysis to produce workflow output artifacts that satisfy various levels of provenance. CWLProv is based on open source community-driven standards, interoperable workflow definitions CWL, structured provenance using the W3C PROV Model [6], and resource aggregation and sharing as workflow-centric Research Object (RO) [16] generated along the final outputs of a given workflow enactment.

Governance in DLTs, Blockchain and Hybrid Blockchains can be described as follows [49]:

To meet the different requirements for applications and companies, in the recent years, different types of blockchains have been developed. Data stored in the blockchain can vary in importance and sensibility and thus it is important to control the access to this information. For example, Bitcoin blockchain is a public and permissionless blockchain that allows anyone to read and send transactions to the network. A permissionless blockchain means that everyone can contribute to the consensus process to validate new block of transactions to be added to the blockchain.

Open networks must use proof-based algorithms to establish trust because members of the network are inherently untrustworthy. Thus, by providing proof that is acceptable to a majority of the remaining network, nodes can be added to the list. Since the proof might be computationally expensive, cryptocurrencies provide the incentive of receiving coins in the currency appropriate to the work.

If the primary motivation of such a costly proof scheme is to create trust between untrustworthy parties, one obvious alternative is to work only with trustworthy nodes. Permissioned networks are composed of nodes identified by cryptographic keys. Other members of the network provide the permission to these nodes to join the network. Consensus is reached in this situation by simply making sure that the source and purpose of the transaction are valid between some parties on the network. There is no need of incentive to check transactions. The simple checking of membership of the parties in the network is sufficient. This improves performance and alleviates any concern about the work, stake, or cost required to participate. Example of permissioned blockchain is the open source Hyperledger Fabric distributed ledger software.

To maintain the reliability and integrity of the blockchain and avoid the recording fraudulent data or double spending transactions, the technology relies on one of its key component: hashing. Hashing is a mathematical algorithm (such as SHA-256 used by Bitcoin) that takes an input of any length and turns it into a cryptographic fixed output. Examples of such inputs can include a short piece of information such as a message or smart contracts, a block of transactions or an address of a content in a content-addressable system such as IPFS.

Consensus protocols are one of the most important and revolutionary aspects of blockchain technology. These protocols create an irrefutable system of agreement between various devices across a distributed network, whilst preventing exploitation of the system.

Blockchain consensus protocols are what keep all the nodes on a network synchronized with each other, while providing an answer to the question: How do we all make sure that we agree on what the truth is?

‘Consensus’ means that the nodes on the network agree on the same state of the blockchain. This, in a sense makes it a self-auditing ecosystem. This is an absolutely crucial aspect of the technology, carrying out two key functions. Firstly, consensus protocols allow a blockchain to be updated, while ensuring that every block in the chain is true and in many cases keeping participants incentivized. Secondly, it prevents any single entity from controlling or derailing the whole blockchain system. The aim of consensus rules is to guarantee a single chain is used and followed.

The InterPlanetary File System (IPFS) is likely the foremost P2P file system at the moment, represents the state of the art in the quest for a distributed web. IPFS takes ideas from previous P2P systems such as distributed hash tables (DHTs), BitTorrent, Git, and and Self-Certified Filesystems (SFS), and tries to simplify those ideas and take them even farther. Nodes in an IPFS network store objects (files and other data structures) in local storage and connect to each other to transfer these objects [50].

Nodes in the IPFS network are identified by a "NodeId", which is the cryptographic hash of a public key. When a node in the network requests a file or other network objects, the network’s routing system finds peers who can serve the requested objects (known as "blocks"), gets the corresponding network addresses, and connects the requester to the discovered peers. When two peers first connect, they exchange public keys and check to make sure the connection is secure. If the check fails, the connection ends. Once the peers have connected, they use an IPFS protocol called BitSwap, which is based on BitTorrent, to "barter" and exchange blocks. This bartering process is meant to prevent freeloader nodes from exploiting the file system. Once the exchange is over, the requester node now has full copies of the blocks it received and can then share them with the next requester [50].

IPFS uses a Mergle DAG (directed acyclic graph) where links between objects are cryptographic hashes of the content the link refers to. This means that all content in the IPFS, including links, is identified by its hash checksum. This, in turn, helps prevent data tampering and eliminates data duplication. It also means that unlike HTTP, which is location-addressed, IPFS is content-addressed.

Objects in IPFS are immutable and permanent, and Git-like versioning is built into the system. Old versions of objects, like every other object in IPFS, can simply be retrieved by their hash checksum.

Although its underlying store is immutable and content-addressed, IPFS does support mutable paths through a decentralized naming system called InterPlanetary Name Space (IPNS). IPNS takes advantage of the mutable state routing system in IPFS to store object hashes as metadata values that point to objects, can be changed, and, if the end user so chooses, can also point to previous versions of the same object. One drawback of the IPNS system is that it does not result in human friendly paths, so another layer needs to be added on top of IPNS if human friendly paths are desired.

BitTorrent File System (BTFS) [51] is both a protocol and network implementation that provides a content-addressable, peer-to-peer mechanism for storing and sharing digital content in a decentralized file system using BitTorrent protocol. BTFS provides a foundation platform for Decentralized Applications, known as DApps.

BitTorrent, the largest P2P network in the world, still relies on centralized torrent file distribution. These torrent repositories are prone to security breaches, outages, and censorship. There have been numerous instances of attacks on torrent hosting web servers reducing service reliability. With a decentralized repository of torrent files leveraging the version control properties of BTFS, users can more reliably access torrent files.

The BitTorrent File System (BTFS), created by the makers of BitTorrent, is an upcoming distributed file system implementation that began as a fork of the IPFS implementation. BTFS promises to take what IPFS has accomplished and present an improved version that is more ready for widespread adoption. The makers of BTFS plan to integrate the existing BitTorrent P2P file exchange system into BTFS, which could help BTFS gain the wide use that IPFS has largely failed to attain [51].

By leveraging the massive existing infrastructure of BitTorrent user nodes (close to 100 million), BTFS is aimed at becoming the largest distributed storage network as well as the world’s largest distributed media sharing network [51].

One of the primary advantages BTFS claims to have over IPFS is its integration of native token economics. These tokens are intended as an incentive for network nodes to contribute by storing data and not simply leeching of existing nodes. BTFS is also set to release with a set of developer tools that will make working with the file system more user friendly than other alternatives. BTFS claims a public version of the system will be available by 2020.[51]

Hyperledger Fabric [52] is an open source enterprise-grade permissioned DLT platform, designed for use in enterprise contexts, that delivers some key differentiating capabilities over other popular distributed ledger or blockchain platforms. Hyperledger Fabric is maintained by the Linux Foundation and evangelized by IBM. Fabric is currently one of the most widely adopted blockchain technologies by the biggest enterprises, much more so than the “big name” blockchains. Companies such as Oracle, Walmart, Airbus, Accenture, Daimler, Thales, The National Association of Realtors, Deutsche Borse Group, and Sony Global Education are all members of the community that develops and maintains Fabric.

One key point of differentiation is that Hyperledger was established under the Linux Foundation, which itself has a long and very successful history of nurturing open source projects under open governance that grow strong sustaining communities and thriving ecosystems. Hyperledger is governed by a diverse technical steering committee, and the Hyperledger Fabric project by a diverse set of maintainers from multiple organizations. Fabric has a development community that has grown to over 35 organizations with nearly 200 developers since its earliest commits.

Fabric has a highly modular and configurable architecture, enabling innovation, versatility and optimization for a broad range of industry use cases including banking, finance, insurance, healthcare, human resources, supply chain and digital product delivery.

"Fabric is the first distributed ledger platform to support smart contracts authored in general-purpose programming languages such as Java, Go and Node.js, rather than constrained domain-specific languages (DSL) and uses Docker container technology for deployment of smart contracts (what Fabric calls “chaincode”)" [52]. This means that most enterprises already have the skill set needed to develop smart contracts with no additional training to learn a new language or DSL is needed.

The Fabric platform is also permissioned, meaning that, unlike with a public permissionless network, the participants are known to each other, rather than anonymous and therefore fully untrusted. This means that while the participants may not fully trust one another (they may, for example, be competitors in the same industry), a network can be operated under a governance model that is built off the trust that does exist between participants, such as a legal agreement or framework for handling disputes.

One of the most important of the platform’s differentiators is its support for pluggable consensus protocols that enable the platform to be more effectively customized to fit particular use cases and trust models. For instance, when deployed within a single enterprise, or operated by a trusted authority, fully byzantine fault tolerant consensus might be considered unnecessary and an excessive drag on performance and throughput.

Fabric can leverage consensus protocols that do not require a native cryptocurrency to incent costly mining or to fuel smart contract execution. Avoidance of a cryptocurrency reduces some significant risk/attack vectors, and absence of cryptographic mining operations means that the platform can be deployed with roughly the same operational cost as any other distributed system.

Hyperledger Fabric delivers a uniquely elastic and extensible architecture, distinguishing it from alternative blockchain solutions. The combination of these differentiating design features makes Fabric one of the best distributed ledger platforms available today both in terms of high degrees of confidentiality, resiliency, flexibility and scalability.

The concept of a Smart Contract was introduced in 1994 by Nick Szabo [53], a legal scholar, and cryptographer. Szabo came up with the idea that a decentralized ledger could be used for smart contracts, otherwise called self-executing contracts, blockchain contracts, or digital contracts. In this format, contracts could be converted to computer code, stored and replicated on the system and supervised by the network of computers that run the blockchain. Contracts would be activated automatically when certain conditions are met.

A smart contract, or simply a contract, in the context of blockchain is a small piece of code that is executed in response to a transaction. Business logic executed on the network is done through contracts. Contracts can have many uses, although some ledgers may limit the type of code that can be executed for either architectural reasons or security. Smart contracts are automatically executable lines of code that are stored on a blockchain which contain predetermined rules. When these rules are met, the code executes on its own and provides the output. In the simplest form, smart contracts are programs that run according to the format that has been set up by their creator. Smart contracts are most beneficial in business collaborations in which they are used to agree upon the decided terms set up by the consent of both the parties. This reduces the risk of fraud, as there is no third-party involved. The costs are also reduced.

In the context of blockchain, a smart contract is defined as machine processable business agreement embedded into the transaction database and executed with transactions. The function of the contract is to define the rules defining the flow of value and state of a transaction needed in business transaction. The contract is smart because it executes the terms of contract using computerized protocol. The core idea behind smart contracts is to codify various contractual clauses (such as acceptance criteria, delineation of property rights, and so forth) to enforce compliance with the terms of the contract and ensure a successful transaction. Smart contracts are designed to guarantee one party that the other will fulfill their agreement. One of the benefits of smart contracts is to reduce the costs of verification and enforcement. Smart contracts are required to be observable (meaning that participants can see or prove each other’s actions pertaining to the contract), verifiable (meaning that participants can prove to other nodes that a contract has been performed or breached), and private (meaning that knowledge of the contents/performance of the contract should involve only the necessary participants required to execute it). Bitcoin has basic support for smart contracts. However, it lacks some essential capabilities such as Turing-completeness, lack of state, and so on.

Launched in 2015, Ethereum blockchain is the world’s leading public programmable blockchain supporting Smart contracts. It replaces Bitcoin‘s more restrictive script language with one that enables developers to build their own decentralized applications. On this platform, smart contracts are implemented with the Solidity scripting language, which is Turing Complete. Solidity is currently the most prominent public smart contracts framework that allows anyone to write smart contracts and decentralized applications by creating their own arbitrary rules for ownership, transaction formats, and state transition functions.

In the open source Hyperledger Fabric, smart contracts, a.k.a. ChainCode program can be written in Go, node.js, or Java. Chaincode is installed on peers and require access to the asset states to perform reads and writes. Chaincode runs in a secured Docker container isolated from the endorsing peer process. The chaincode is then instantiated on specific channels for specific peers. Chaincode initializes and manages the ledger state through transactions submitted by applications. A chaincode typically handles business logic agreed to by members of the network, so it similar to a “smart contract”. A chaincode can be invoked to update or query the ledger in a proposal transaction. Given the appropriate permission, a chaincode may invoke another chaincode, either in the same channel or in different channels, to access its state. Note that, if the called chaincode is on a different channel from the calling chaincode, only read query is allowed. That is, the called chaincode on a different channel is only a Query, which does not participate in state validation checks in subsequent commit phase.

Smart contracts are one of the most successful applications of blockchain technology. Using smart contracts in place of traditional ones can reduce the transaction costs and trusted intermediaries significantly and improve automation and security. They are also tamper-proof, as no one can change what has been programmed.

The Internet was built without an Identity Layer. That is, there was no standard way of identifying people and organizations. The addressing system was based solely on identifying machine physical endpoints, not people or organizations. In his excellent article “The Path to Self-Sovereign Identity” [54], Christopher Allen provides a clear analysis of the online identity landscape and describes the evolutionary path composed of 4 broad stages since the advent of the Internet: Centralized identity, Federated identity, User-Centric identity, and Self-Sovereign identity (see Figure 4).

The evolution of internet identity is the result of trying to satisfy three basic requirements [55]:

The following describes the different evolution stages of identity model in more detail.

Public Key Infrastructure (PKI) is a system of hardware, software, people, policies, documents, and procedures. PKI includes the creation, issuance, management, distribution, usage, storage, and revocation of digital certificates. These certificates are then used to authenticate the identities of various actors across the data transfer process. They also assure that the data being moved between these actors is secured and encrypted in a way that both parties can decrypt. This way, information is only being sent to and received from known and trusted sources, and both parties are assured of the information’s integrity.

Currently the most commonly employed approach for Identity Management is based on Web PKI, a centralized trust infrastructure based on public key infrastructures (PKIs). Web PKI is a Certificate Authority (CA) based system that adopts a centralized trust infrastructure. Communications over the internet are secured through the safe delivery of public keys and the corresponding private keys.

PKI trust is established by a certificate authority, which is an organization or governing body that can issue certificates and verify the identity of the certificate requestor. For example, AWS offers multiple PKI certificate authority services that can help organization to easily and securely manage their certificate infrastructure.

The open source Hyperledger Fabric, which is used to implement permissioned blockchains, adopts the traditional PKI hierarchical model. Identity verification come from a trusted authority called membership service provider (MSP). An MSP is a component that defines the rules that govern the valid identities for this organization. For example, the default MSP implementation in Hyperledger Fabric uses X.509 certificates as identities. MSPs turn verifiable identities into the members of a blockchain network.

The following describes, in more detail, the four key elements to PKI:

In 1995, Phil Zimmerman proposed the encryption program Pretty Good Privacy (PGP) [60], which is a decentralized trust system that was created before blockchain existed. However, PGP has issues with establishing trust relations between all parties. But today, with DLTs technologies, there is no need for the third-parties. The emergence of distributed ledger technology (DLT) and blockchain technology provides the opportunity for fully decentralized identity management. DLTs provide a new approach to building a more competent, secure PKI system. Decentralized Public Key Infrastructure, or DPKI, is an alternative approach to designing better PKI systems. In 2015, Allen et al. [59] asserted that unlike the traditional approach, DPKI ensures no single third-party can compromise the integrity and security of the system as a whole. DPKI could have a bigger impact on global cybersecurity and cyber-privacy than the development of the SSL/TLS protocol for encrypted Web traffic (now the largest PKI in the world).

In DPKI, trust is decentralized through the use of technologies that make it possible for geographically and politically disparate entities to reach consensus on the state of a shared database. DPKI is based on Distributed Ledger Technologies (typically blockchain key-value datastores). In a decentralized identity system, entities (discrete identifiable units such as, but not limited to, people, organizations, and things) are free to use any shared root of trust. Globally distributed ledgers, decentralized P2P networks, or other systems with similar capabilities, provide the means for managing a root of trust without introducing a centralized authority or a single point of failure. In combination, DLTs and decentralized identity management systems enable any entity to create and manage their own identifiers on any number of distributed, independent roots of trust [61].

In blockchain-powered DPKI, the new third parties become miners or validators. The trust is established and maintained based on consensus protocols. The miners or validators will have to follow the rules of the protocol, that would financially reward and punish these third-parties to effectively preventing misbehavior in the blockchain and limiting their roles. Bitcoin, devised by Satoshi Nakamoto, is the first such successful protocol. It is based on proof-of-work, in which the energy expenditure of "miners" is used to secure the database. Other consensus protocols could be used such as proof-of-stake (PoS).

Figure 12 illustrates in a nutshell, the difference between PKI and DPKI. In a PKI based system Alice and Bob need to establish a way to exchange and store their public keys. Conversely, in a blockchain-based web of trust model, the storage of public keys is managed on the public ledger. As participants in a global identity network, Alice and Bob create their unique DIDs, attach their public keys and write them to the public ledger. Now any person or organization that can discover these Decentralized Identifiers (DIDs) will be able to acquire access to the associated public keys for verification purposes.

There are two emergent and important standards that play a central role in the enablement of DPKI, the W3C Decentralized Identifiers (DIDs) [61] and the W3c Verifiable Credentials Data Model [62]. The next sections, which are derived from the W3C Primer for Decentralized Identifiers [63], describe these foundational specifications in more detail.

The number of possible connections between any given set of entities in the cloud is an impossibly large number. Human, machine, or software agents increasingly need to interact in a trusted manner with a different group of these interconnected objects to achieve their goals in both the digital and physical worlds. This requires a straightforward and ubiquitous method to address, verify, and connect these elements together. Unfortunately, the current centralized PKI approach is not able to scale with the growth of IoT and has severe usability and security limitations. The Internet does not have a built-layer for managing identity and trust in decentralized way. Human or object identities are stored in multiple centralized or federated systems such as government, ERP, IoT, or manufacturing systems. From a cryptographic trust verification standpoint, each of these centralized authorities serves as its own root of trust. An entity trailing along a value chain is interacting with multiple systems. Consequently, a new actor in any given value chain has no method to independently verify credentials of a human or attributes of either a physical object or data item (provenance, audit trail). This results in the existence of complex validation, quality inspection, and paper trail processes, and enormous hidden trust overhead costs are added to all value chains and services [67].

The solution outlined in this section describes the general principles that needs to be adhered to accomplish the goal of establishing a web of trust. The solution is not exhaustive and complete as some important standards still need to be defined by the community.

The proposed solution to address the challenges outlined in the previous section is based on DLTs and a new layer of decentralized identity infrastructure, which provides a straightforward, ubiquitous, universal method and associated interoperable protocols to address, verify, and connect all the entities participating in complex workflow ecosystems. Entities include living organisms, physical objects, locations, events, machines, IoT devices, digital assets, datasets or agents. The solution is based on the emerging Decentralized identifier (DID) standard supported by W3C and the Decentralized Identity Foundation (DIF), which defines a new and open standard type of globally unique identifier that offers a model for lifetime-scope portable digital identity that does not depend on any centralized authority and that can never be taken away by third-parties [61]. The core idea of the approach is to use DIDs to identify every entity involved in the complex workflows. DIDs will identify and manage objects, machines, or agents through their digital twins and will be expanded to digital assets, locations, and to events. Data assets are referred to as Decentralized Autonomic Data (DAD) items [68],[67].

The objective of the suggested approach is to capture unforgeable audit trail referring to the DIDs of all the entities involved in each step of the processing workflow. The data flow provenance is captured via a self-referential blockchain, which consists of a list of signed DAD items where each subsequent DAD item includes the DID(s) or dDID and associated signature of the previous item, forming a Merkle tree.

DID can utilize the W3C Verifiable Credentials standard to prove verified aspects of their identity when signing claims about other DIDs. Such a use case could enable verified organizations to sign a claim with their DID that they have produced a digital asset according given specifications, certify software, service, hardware and standard-based provenance document that could be reused to reproduce the data asset.

By integrating decentralized identifiers, verifiable claims, DLTs, federated analytics can be achieved in a fully decentralized, peer-to-peer collaboration mode, without including a third party to coordinate the process. Everything is trusted, transparent and traceable abd organizations can now control their own digital identities, rather than relegating control to their vendors. This increases security, accuracy, and trust for all participants in the global supply chain.

A paradigm shift is now occurring with the exponential growth of Big Data due to the rise of the Internet of Things (IoT), social media, mobility, and other data sources, as it defies the scalability of centralized approaches to store and analyze data in a single location. Participants in complex federated analytics workflows may include, not only different organizations, but also devices from the Internet of Things (IOT). To be able to trust each participant of the workflow, it is necessary to have a scalable, secure and robust identity management solution. As discussed in the Identity Management section, centralized identification systems based on PKI suffer from serious security and usability challenges. Testbed-15 proposes a solution to address this challenge is to use a DPKI infrastructure leveraging DLTs and Blockchain to build a Web of Trust without centralized root of trust. To bootstrap the web of trust between participants, each participant and artifacts produced by the workflow should be assigned with a resolvable, permanent decentralized identifier using the DID. The DID would be registered on a blockchain using a method specification that allows the storage of the DID document on the DLTs. For example, IPFS can be used as a method to store the document in a distributed way. The access of the document would be resolvable anytime from anywhere due to the replication of the document over the P2P network. The DID of each participant would be used to publish and sign verifiable claims using the W3C Verifiable Credentials standards.

Within the solution describes above, DIDs would also be used to reference artifacts and provenance information produced by federated analytic workflows. By doing so, they can be referred to in verifiable claims that can provide information about the author, provenance information, time stamps and certification information. The credential data model authors are expected to use machine-readable vocabularies through the use of Linked data ontologies (for example schema.org) that the different parties of the federated cloud agree upon.

A DID resolver is a software or hardware component with an API for resolving DIDs of at least one DID method. It executes the read operation for the DID method corresponding to the DID being resolved to obtain the authoritative DID document [61].

The DIF Universal Resolver supports different “methods” for registering identifiers on decentralized systems such as the Bitcoin blockchain, Sovrin, Ethereum, IPFS, and others. It provides a unified interface which can be used to resolve any kind of decentralized identifier. This enables higher level data formats (such as Verifiable Claims) and protocols (such as DIF’s Hub protocol) to be built on top of the identifier layer, no matter which DLT, blockchain or other system has been used to register the identifier. Internally, this is achieved by the use of “drivers” for each supported identifier type. A driver can be easily added via a Docker container, a Java API, or a remote HTTP GET call, as illustrated in the following diagram Figure 22:

Currently a number of driver specification documents have been defined for Sovrin, Bitcoin, Blockstack, uPort, Kolocom, IPFS, IPNS and Veres One Distributed Ledgers. Drivers can internally function in different ways — some drivers may simply call a remote web service (can make the driver usable even on mobile devices), while others may have direct access to a full node of the blockchain they use which provides higher security guarantees for the resolution result.

Shawn Conway introduced in his seminal paper "A DID for Everything" [67] a mechanism to capture attribution, verification and provenance for entities and Data Items using DADs. "The provenance of data in a data flow through a processing system that transforms data can be established by forming a literal block chain of the data. When using DAD items to represent the data, the chain of DADs can be represented simply in a self-contained manner. At each step in the data flow of the originating DAD, where the contained data is transformed in any way, a new DAD is generated by the controlling entity of the transformation. This entity assigns a new DID or derived DID (dDID) to this DAD. The new DAD payload includes the DID(s) of the DAD prior to transformation as well as the signature of the prior DAD. This links the new DAD to the prior DADs. The signature of the prior DAD provides both a hash that establishes the content integrity of the prior DAD as well a non-repudiation of the controller of the embedded prior DID. All the DADs in the data flow need to be stored some place indexed by their DIDs. Given this storage, any single DAD can then be used to recall the string of prior DADs back to the originating DAD or DADs. A special case is when an entity merely wishes to establish custody of data without changing or transforming it. The simplest way to do this, is for the entity to add a copy of the DAD as a link in the DAD chain without changing the underlying data. This forms an assertion that they control that link. If they do not transform the data then merely signing is enough to assert control over the link or equivalent custody of the the data for that link of the chain. A one to many data stream is just a branch or fork in the chain into multiple chains." [67].

Figure 23 illustrates a complex Directed Acyclic Graph (DAG) Decentralized Autonomic Data Flow.

Provenance of both the controller of the transformation step and the integrity of the associated data can be determined because each DAD embeds a DID and is signed by the associated private key belonging to the DID. Full traceability can be established back to the originating DAD or DADs, preserving both data integrity and proof of control because each DAD embeds a DID and is signed by the associated private key belonging to the DID. This is a critically enabling capability for decentralized computing infrastructure [67].

The data provenance and its corresponding source data might be critical; therefore, it must be available at anytime from anywhere.

Why is this important in the context of this study? One of the main reasons is to have robust and permanent availability of provenance information. IPFS is an example of Content-addressable storage that could be used for storing such information. The content could be encrypted to control the access of the information when sensitive information needs to be protected. As long as the data have been duplicated on other peers of the network, the information will remain permanently on the network and can be accessed anytime by using its content hash address.

The W3C published a number of use cases for Verifiable Credentials [69] related to education, retail, finance, healthcare, professional credentials, legal identity and devices. However, there is no use case related to supply chain of physical and digital assets. Each variable credential is composed of a series of claims (i.e., RDF statements) asserted by an authoritative entity (the Issuer) about another entity (the Subject), owned by yet another entity (the Holder who most of the time is identical to the Subject but not always) and verified by a final entity (the Inspector/Verifier). Using the W3C Verifiable Credentials standards, verifiable claims can be either self-issued by an entity such as a machine to provide a proof about authenticity and integrity of data or they can be issued by a third party. Verifiable Credentials will automate authentication and authorization for consumers and consequently reduce the cost of service delivery for providers. Verifiable Credentials are claims made against an identity (DID) representing an asset such as an algorithm, machine, certificate of authenticity, provenance documentation, etc. that can be verified by a third party verifier. These claims are cryptographically secure, privacy-respecting and machine-verifiable.

In federated cloud systems, any node might want to transact with any other node, engaging with each other in a dynamically defined, on-demand way. To ensure efficient transactions any new entity involved in a value chain must be able to independently verify other counter parties. This is achieved by using DIDs which anchor the verifiable claims on a distributed ledger technology, so they can move the cryptographic root of trust from a centralized to a decentralized, interoperable infrastructure.

Because the claims are expressed in Linked Data, parties sharing credentials need to have a common understanding of the vocabulary used to make the claims. The Verifiable Claim needs to refer to a schema (ontology) that is agreed by different parties. Example of ontology is schema.org or OpenBadge, which could be used to make assertions about people and other web assets. One important aspect of relying on schemas to provide the semantic meaning of data within a verifiable credential, is that the meaning of the credential properties should not change. It is not enough for entities within the ecosystem to have a shared understanding of the data in the present, it may be necessary for them to have an understanding of the credential at the time it was issued and signed. This depends on the trust framework within which the credential was issued and the needs of the parties involved. A verifiable data registry can provide immutable storage of schemas.

The storage of information in a blockchain network is an important issue to be considered from an identity management, asset management, scalability, privacy, security and cost. There are concerns about storing private data (even encrypted), and there is a need to avoid potential security and privacy problems due to possible vulnerability to advanced quantum machines in the future. However, many of the available blockchain platforms do not require private data storing on a blockchain. Another concern is that some private data security approaches might compromise the scalability of the blockchain network. The preferred solution is therefore to use blockchain to store and lookup decentralized identifiers (or their hashes) of the assets, organizations, agents, services, people, and consent of data sharing between the asset owners. Since the proofs of data are stored on the blockchain rather than the identity/asset data themselves, the scalability, cost and storage size are not operational challenges. As a result, private data of any kind (including hashed personal data) and private proof of existence are generally speaking not stored on the ledger.

There are several projects currently ongoing to explore Earth Observation (EO) data provenance and traceability using blockchain technology. One of them, which implemented as a proof-of-concept, has been conceived at the European Space Agency (ESA) and aims to develop and prototype a set of new technologies to enable secured and traceable exploitation of data from space missions focusing on the EO segment. The high-level objectives of the ESA data provenance project included:

The technical implementation of the ESA data provenance project consisted of several steps: a survey of state-of-the-art DLT technology and analysis of potential requirements; design and development of trusted data sharing processes for the supply chain including necessary software components; integration of the developed software onto the selected data acquisition and distribution infrastructures; and validation and demonstration on identified use cases. The implemented solution provided the concept of a provenance chain which relies on hashes calculated from EO products and processors using the KSI (Keyless Signature Infrastructure) - a blockchain technology designed for automated verification of digital signatures based on record of data integrity and time. KSI only relies on cryptographic properties of hash functions and the availability of widely published verification codes which are stored in the Guardtime blockchain [70]. The results of the ESA data provenance project, when published, could inform future work by OGC testbeds.

Future works on blockchain, DLT and provenance for federated clouds should focus on the following aspects, among others: