OGC Testbed-15: Federated Clouds Security Engineering Report

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

While federation can mean different things based on different perspectives, some definitions have been collected which address different facets of federation:

Notably, these definitions in increasing conciseness go from "interaction or collaboration of some sort" to "agreement on standards of operation" and finally concrete service properties.

During Testbed-15, the following (non-exhaustive) list of federation facets emerged:

Instead of providing its own definition of Federation, this document puts forward the different approaches to Federation, commonly named Federation Facets. These Facets have in part also been identified in previous OGC Innovation Program Testbed activities. As this ER is heavily focused on the Security aspects of Federation, some of these layers may be detailed in a greater extent than others.

Please note that the concepts of Federation and Security do not necessarily need to coexist within specific environments, since this will depend on which approach is implemented in a Federation Layer that is being utilized by the Administrative Environment that deploys the Federation components. Nevertheless, any Federation Architecture should have its own embedded Security Architecture.

Some of these approaches for implementing Federation Layers are directly tied to specific Security topics:

Although other Layers are mentioned in this section, the focus of this ER is on the Identity and Resource Management and the security use cases that they enable.

The concept of federation entails much more than the transparency and sharing of resources that happens at the end-user level. In most cases, the creation of a Federation requires several high-level entities to agree on how these resources are shared, who is their owner, and which are the basic rules and requisites to be part of the federation.

A governance model represents all these agreements (which in themselves are the governance framework) and policies. Therefore, a governance model allows clearly defining all entities, practices, and procedures within the federation. These governance models, when put in practice, result in several deployment models that reflect the degree of openness, the possible stakeholders and their capability to influence the federation.

During the Testbed-15 activity, an effort was made to demonstrate the complexity and issues tied to the concept of Federation. This was not only between Services/Resources and the Federation but also the interaction between Federated environments (also known as Federation of Federations or inter-federation). To do so, the concept of Federation Manager is put forward and explored in detail in the following sections.

The main objective was to simulate a scenario whereby two Federated Environments interact with each other employing their Federation Manager components and expose their endpoints to provide membership, resource access, and policy management capabilities. These capabilities, from the point of view of the Security domain, provide the necessary layers of Authentication and Authorization. In addition to this, each Federation Manager internally handled one or more services. Both Federation Managers represent different levels of maturity in different areas. This adds to the complexity of making them interact with each other.

The NIST Cloud Federation Reference Architecture [3] puts forward the following definition of Federation Manager:

In general terms, a Federation Manager should aim to produce the most matured Federation possible. Maturity of a federation, in this case, is measured by how much of the governance work is automated (leaving only the remaining stakeholder human interactions required to solve governance issues) and the degree of Security achieved in their interactions when providing the Access Control and Identity capabilities.

Taking this definition into account, several functionality aspects are required from a Federation Manager. To enable inter-federation, these aspects need to be served not only to users but also to other Federation Managers.

The NIST Cloud Federated Clouds Reference Architecture [3] identifies several Federation Manager (FM) deployments, that assume a series of properties mainly based on their location within their respective Federation environments, and their interactions with other Federation Managers.

From the point of view of Service Providers, any Federation Manager is expected to provide the means to efficiently interact with End-Users. This means providing Authentication and Authorization capabilities as well as interaction with the Resource Owners who contribute with their services in the Federation.

Ideally, a Federation Manager should be as invisible as possible to clients while enabling the use of dedicated protocols and APIs for the Service Providers (and clients) to fulfill the use cases listed in the following sections, for all stakeholders. Bear in mind that the role of Resource Owner could represent different stakeholders depending on the Governance Model as stated in Governance is important.

In addition to the previously mentioned use cases, a Federation needs to efficiently communicate with other Federations, thus enabling inter-federation. Both membership attributes and resource metadata need to be shared and modified in a controlled manner to achieve convergence between any number of Federation Managers.

The amount of expected functionality from a Federation Manager increases the complexity of previously tested deployments (such as the one utilized in Testbed 14 [5]). The structure for Federation Manager #1 is shown in the following figure:

Federation Manager #1 utilizes state-of-the-art standards to provide Authentication and Authorization capabilities using:

A break-down of all components and the use cases enabled by each of them is provided in the following sections.

On the other hand, three main types of Client Applications are identified based on their interactions with the Federation Manager endpoints and the End-Users they support.

This section focuses on the use cases mentioned in Federation and how the Federation Manager aims at fulfilling them.

The focus of this deliverable was to protect resources that are highly tied to REST API implementations. For example, web service implementations of the emerging OGC APIs that host features, processing tools, coverage files, or any other resource made available through a REST API. The applicability to web services makes it more scalable than other resource management options such as certificate signing, which as key factor on its viability as a Federated Clouds solution.

The API endpoints are also potentially usable in more operational scenarios where access to stored datasets needs to be protected (such as S3 interfaces that sit in front of data stores or "data lakes"). Please note both protected APIs (for Membership and Resource Management) can be leveraged in M2M (Machine-to-Machine) interactions allowing higher levels of automation and maturity of the federation. Further, this allows the creation of complex, cross-domain workflows with components that reside in independent security domains.

Future work should explore the situation whereby an external federation acts as an external resource by treating its users and resources as if they were already in the original Federation. In this case, the administrator would act as a Resource Owner, restricting the external visibility of his resources.

The final added value for this deliverable resides in the ability to allow external Federation Managers to consume these services and to at least provide a governance model that allows all NIST deployment models to be tested [3].

For the purposes of Testbed-15, a minimal demonstration was made available as an ad-hoc service to the Federation Manager, giving access to as much of the functionality mentioned in this section, and the usage of personal and intermediate X.509 certificates has also been demonstrated in the Technology Integration Experiments (TIES).

The rasdaman datacube analytics engine was the Federation Manager (FM) #2 for the Testbed-15 federation experiments. The engine uses a location-transparent distributed database. For Testbed-15, the engine was coupled with the Deimos FM #1 described above. In this section the Federation Manager #2 and its inter-federation with FM #1 are described.

As Testbed-15 addressed data and service offerings (cf. definition of Data Federations and Service Federations earlier) considering existing work is adequate. Federations have been discussed for decades in Distributed Database Systems. A compact definition of a distributed database is: "A distributed database management system (DDBMS) is a set of multiple, logically interrelated databases distributed over a network. They provide a mechanism that makes the distribution of data transparent to users." (Techopedia) Among the "mechanisms that makes the distribution of data transparent" are means to

Such functionality is closely related to Distributed Databases, and suggests a discussion regarding similarities and differences of "federated vs distributed databases". As a starting point, the "border" between both architectures is fuzzy. Basically, a distributed database is managed by a database controller (just one), has one overarching schema, and hence forms a logical and functional unit. A federation of databases instead involves a coupling of independently existing (and evolving) databases, without centralized control.

For example, in a distributed relational database the sub-tables of a distributed table cannot change their schema (such as list of attributes) unilaterally. In a federation, data can evolve independently as each node retains full autonomy. The analysis performed during Testbed-15 found that, regarding access control, operators generally prefer local autonomy in database design and access control. However, this ER acknowledges that this approach is empirical and may not apply to all data centers and their policies.

rasdaman is different from the FM #1 architecture in that a built-in, tightly integrated federation manager component is available. This section briefly describes the rasdaman architecture. More details and Big Earth Data application scenarios can be found in [7].

The rasdaman engine resembles a full-stack Array DBMS where each component is designed and optimized for management and retrieval of massive multi-dimensional arrays. The raster query language used by rasdaman, rasql, extends SQL with declarative n-D array operators. As such, rasql forms the blueprint for "Array SQL", ISO SQL 9075-15:2019 Multi-Dimensional Arrays (MDA).

Tiled array storage can be configured for some relational database system (such as PostgreSQL), directly in some file system (which is about 2x faster than using a DBMS), or by accessing pre-existing archives without copying data into rasdaman. A versatile intelligent ETL tool homogenizes data so that within each datacube there is one common coordinate reference system, resolution (if regularly gridded), and null value set thereby also applying defaults, responding to missing data and metadata, and so forth. Likewise, internal database tuning can be performed by the rasdaman database administrator, including adaptive data partitioning ("tiling"), lossless and lossy compression, indexing, and cache sizing.

Incoming queries undergo a variety of individual optimizations, including query rewriting, common subexpression elimination, cost-based optimization, join strategy selection, multi-core parallelization, semantic caching, etc. Queries can be split over multiple nodes in a cloud for efficient parallel evaluation. By using the same principle of query splitting not only within clouds, but also across data centers, datacube federations can be established. In such a rasdaman federation, worker processes can fork subqueries to other nodes for load sharing and data transport minimization. The Figure below illustrates how expressions submitted by a client get split and dispatched according to data locations ("ship code to data"). The illustration presents a query that has been successfully distributed across more than 1,000 Amazon Web Services (AWS) cloud nodes. This is described in more detail below.

The rasdaman system described so far is domain independent and can just as well serve and analyze cosmological simulations, gene expression, fintech, and business data, to name a few examples. Spatio-temporal semantics as per OGC CIS are added using an extra layer with information on coordinate reference systems (CRS) as well as regular and irregular grids, and so forth. This service interface implements OGC Web Map Service (WMS), Web Coverage Service (WCS), and Web Coverage Processing Service (WCPS). Spatio-temporal coverage functionality ranges from simple subsetting and re-formatting to complex datacube analytics.

In a rasdaman federation, the user sends a query to one of the rasdaman servers, which in turn might send subqueries to different federation partners. The rasdaman federation is location transparent. Queries can be sent to any federation member, and this member will orchestrate automatically with all members as needed (i.e. holding relevant data) to jointly process and return the query result. There is no central federation manager, rather a distributed federation mechanism is integrated with each server node. Technically, this is similar to a DNS: the network permanently updates itself and each node propagates information on changes. Hence, every node knows the global situation at any time (modulo networks latency) - knowledge that is also used for query optimization. Exchange turns out to cause negligible additional network load. Nodes still know what their local data and what remote data are, and this is communicated to users via the standard Capabilities document. The advantages of such an approach are:

A basic principle of a rasdaman federation is that Authentication is global while Authorization is local. Global authentication - either via the built-in rasdaman authentication manager or some external authentication server - ensures that query fragments forwarded to federation partners will be accepted by those. Authorization is strictly local - only the local administrator can define roles, but no external entity (such as a separate global federation manager). This maximizes security while leaving enough flexibility in distributed query processing in combination with individual data policies, protecting down to the level of single pixels [8].

A federation is established by service owners who agree on mutual trust when federated queries are involved, combining own with federation member data for query answering. This is a decision made by humans, reflected by:

In the extreme case, a single administrator with full power over all member nodes may determine all these settings across the complete federation. At the other end, authorization and access control remain fully local with each node, with full autonomy retained. While a coordinated approach technically is not necessary some level of coordination appears feasible in practice as the resulting network easily can become unmanageable as it grows. For example, disallowing access to objects advertised does not make much sense and should be avoided - one reason why a federation should be based on some trust and agreement among the federation operators. In the end, however, the degree of centrality vs distributed management is simply a matter of configuration. In summary, the mechanics (trust via mutually accepting the other fed members, secured communication, common identity management, etc.) in rasdaman is provided by the software, the governance model on top of this is up to the federation operators.

Establishing such governance models requires human agreements. The participants in this Testbed activity recognized that the decision whether to join a federation becomes an executive topic, involving rules & procedures that are deemed acceptable. In the emerging European Datacube Federation, a charter is being elaborated establishing common understanding and governance principles as the "human facet" of the federation. A General Assembly with one vote per data provider will decide about acceptance of new federation members. The goal is to provide governance rules which a concrete federation may accept, or adjust, or write from scratch. Of course, upon agreement of all parties with executive federation power the rules can be changed during the lifetime of the federation.

In Testbed-15, FM #2 has been instantiated with a rasdaman federation combining several AWS cloud nodes allocated in the US with nodes of the emerging European Earth Datacube Federation. Queries can be sent to any node while freely addressing and combining data from anywhere in this federation.

Based on this starting pint as FM #2, in Testbed-15 a hierarchy of federations has been experimented with combining two basically "flat" federations consisting of FM #1 and #2 as a novel idea. Due to the different approaches of the heterogeneous combination (such as location-transparency vs location-awareness) the combined federation will not likely be able to offer the union of all features, but at best the intersection. In case of FM #1 and #2 this was a common authentication scheme based on a single identity manager to which FM #2 "outsources" authentication. Notably, Testbed-15 was repurposed upfront with an asymmetric approach in mind where FM #1 offers services to be consumed by FM #2, to compensate for the fact that this behavior could not necessarily be reciprocal.)

Common role-based access control (RBAC) governs authentication and authorization. Named users are associated with roles that can be recursively nested with sub-roles. As an innovation, triggers protect datacubes from all access. A trigger is defined by the administrator as a constraint on a datacube that any query must fulfill, such as “this area within the datacube must not be accessed” or “at most N pixels may be accessed by a query”; all triggers applying get evaluated dynamically on incoming queries acting like a guardian or a watchdog on “their” datacube. However, users and roles can get exempted from such a guardian so that queries can “pass”, thereby establishing privileges for particular users.

This mechanism establishes mandatory access control: data sets by default are protected against all users (including those getting defined in the future) and any exemption is introduced explicitly by the administrator. This way there is a guarantee that triggers defined on a data set need explicit exemption and cannot be missed accidentally, which would create security vulnerabilities.

Triggers are defined through some query expression which allows for protection of the complete cubes, sub-boxes, polygonal areas, or masks defined through other data sets or algorithmically, down to the level of single pixels.

A user of the federation is any local user of any server participating in the federation. Alternatively, this could be a user maintained by an external trusted Identity Manager which rasdaman has been configured to accept. The server managing a user’s information is called the user’s host.

In Testbed-15, a rasdaman federation was established combining several AWS cloud nodes allocated in the US with nodes of the European Earth Datacube Federation containing in particular a DIAS and CODE-DE, the German Sentinel hub, providing WCS, WMS and WCPS access to Sentinel 2, Temperature and Precipitation datasets. The federation served users of both FM #1 and FM #2, and access control rules can be set for both categories. An external entity, with appropriate privileges, could manage authorization within the federation realm of FM#2, although FM#1 did not exploit this in Testbed-15. Authentication was performed via the identity provider of FM#1.

When visiting the FM #2 endpoint, the user was greeted by a log in screen, where a choice of local authentication (for the FM #2 users) and external authentication (for the FM #1 users) was available. Authentication of external users (users of FM #1 accessing data managed by FM #2) was accomplished by implementing an OAuth authentication flow against the endpoint provided by FM #1. Users choosing to authenticate using OAuth were redirected to an FM #1 page where they can log in using their FM #1 credentials. Upon successful authentication, FM #1 generated an authorization code and redirected the user back to FM #2. FM #2 then uses the authorization code for requesting an access token from FM #1.

After the authentication step, access control rules were established for both FM #1 and FM #2 users using the mechanism described in the previous section. The admin can distinguish users based on the authority issuing the access token (DEIMOS for FM #1 and rasdaman for FM #2). The admin can also assign general roles to all users issued by the same authority, as well as individual roles for each user taking part. Then exemptions from trigger compliance preventing access in particular cases can be issued for any of these roles.

For example, consider a situation where a trigger that prevents access to a secret area of the Sentinel 2 True Color dataset was established by the FM #2 admin. An exception from the trigger is then created for the role assigned to all FM #1 users. The net result is: No user of FM #2 can access the secret area, but all users of FM #1 can. Conversely, an exception from a trigger preventing access to a secret area of Sentinel 2 NDVI was granted to the role assigned to all FM #2 users, making this area available to local users only (an FM #1 user trying to access the area would receive an Access Denied error). Further triggers and exemptions were defined based on finer grained roles of local FM #2 users. In the end, in the federation managed by FM #2, there are several types of datasets (or parts of datasets):

Relying on industry standards like Role-Based Access Control, SAML-based authentication, etc. allowed for a relatively straightforward connection of the rasdaman federation with FM#1, as described. The tight integration of all components in rasdaman minimizes complexity for the administrator while offering a maximum of flexibility in policy configuration. That said, the security configuration spectrum possible, from centrally managed federations to completely autonomous federation members, may be nontrivial to oversee for federation administrators.

Consequently, future testbeds should look to define governance rules for federations, including prefabricated specializations for standard situations such as: scientific open services; scientific open services with embargos; commercial services with paid access; tightly protected services and private federations. Goal is to provide best practice solutions for establishing federations to (i) ease setup and management of federations and (ii) safely establish and maintain a defined security level. In parallel to this conceptual work an aggressive growth of the existing federations is under work, with an eye on merging them where the federation policies are sufficiently compatible.

Overall, the diversity of federation approaches in Testbed-15 has surfaced important issues on service federations in general and has led to initial insights. Insofar, Testbed-15 has kicked off charting the new field of federation management.

This section contains the list of components that participated in TIE execution during Testbed-15.

It is important to take into account that due to the fact that D144 and D147 are highly coupled:

The following matrix focuses on the use cases enabled by each of the Deliverables. These use cases are extensively defined in management, and server to refine a final TIE list with viable functionality tests.

The tests could only partially rely on OGC W*S services as not all FMs offer interfaces accessible through default clients. FM #1 requires a separate access step whereas FM #2, due to its integrated federation manager, allows standard clients like Leaflet, ESA-NASA WebWorldWind, python, and R to use the vanilla OGC WMS, WCS, and WCPS functionality.

*Provided by the FM Proxy component

The table shows two Federation Manager components able to provide their federation environments with all the internal functionality identified in this ER. Nevertheless, they fail to fulfill the purpose of inter-federation with each other due to several reasons:

Following the strategy defined in the Testbed-15 roadmap a "one-way" Federation was established where FM#2 connects with the authentication functionality provided by FM#1, and to provide an approach to solve the differing approaches between both that facilitates future TIEs that aim at tackling the rest of use cases not covered in the pairings of this Testbed, assessing the various options and facets elaborated in this experiment, and ultimately advancing interoperability of federations.

The following TIE tests have been performed:

Two federation managers with differing approaches have been inspected and paired in Testbed-15. FM#1 is a standalone component offering authentication and authorization to which nodes can subscribe; its service remains visible to clients which need to approach FM#1 first, and then approaches the target service as usual. FM#2 is tightly integrated in a fully distributed, location-transparent service, and clients accessing the service do not need to perform an extra step when invoking the federation service. Both architectures lead to different technical consequences which have been investigated and described in this report. As such, the report forms a suitable basis for further investigation on the interoperability between and across federations.