OGC Testbed-14: Secure Client Test Engineering Report

The requirements addressed by this ER are to document the following two use cases:

OGC Testbed-14 developed a client validation engine (client validator) to:

The OGC Web Services Security Candidate Standard defines an annotation mechanism for Capabilities documents or responses to the GetCapabilities request, ensuring interoperability between a secured OGC Web Service (OWS) instance and a client application. It further overrides existing HTTP protocol limitations and exception handling from existing OWS standards for the purpose of achieving interoperability with mainstream Information Technology (IT) security standards and their implementations. The standard has been informed by security-related work in OGC Testbed-11 [1], OGC Testbed-12 [2], and OGC Testbed-13 [3].

This ER extends the OGC Web Services Security Candidate Standard to allow for proper specification of different authentication methods.

The ER stimulates future work regarding OGC standardization requirements for client applications to work on modern security aspects. More details can be found in the section titled "Summary of Findings".

All questions regarding this document should be directed to the editor or the contributors:

Contacts

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The Open Geospatial Consortium shall not be held responsible for identifying any or all such patent rights.

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the standard set forth in this document, and to provide supporting documentation.

To validate a secure client’s conformance to the standard, an executable test suite that can act as a server must be developed. Guidelines on the requirements for such a test suite harness are covered in Annex B of OGC Web Services Security Candidate Standard, and can be summarized as follows:

These test harness requirements must be combined with the goal for OGC Testbed-14, testing a secure client’s ability to adhere to Use Case 1 and Use Case 2 of the OGC Web Services Security Candidate Standard.

The specific method of authentication is not defined, but it could be HTTP Basic or SSL Client Certificates.

Discussion in the CITE thread with other participants led to the selection of two client authentication methods for this implementation. The first is OpenID Connect, and the second is version 2 of the Security Assertion Markup Language (SAML) standard. Details on the implementation of these in the test suite will be explained below, and the testing procedure for these methods described in a later chapter.

Development of the test suite implementation was influenced by existing work from other CITE Executable Test Suites (ETS), particularly the WMS Client test suite. Nearly all other test suites are based on testing a service’s implementation, and those work by integrating a client in the test suite to examine interactions with a service or deployment. For the secure client test, this will have to be inverted, by integrating a server into the test suite to test a client’s interactions.

The WMS Client test suite does this by integrating a proxy service to communicate with actual WMS implementations. By having the WMS client connect to the proxy, the test suite can act as a broker between the client and an actual WMS implementation, while collecting information on the client’s behavior in its HTTP requests. These HTTP requests can then be analyzed by tests to verify their conformance to a standard (in this case, the WMS standard). It is useful to understand how the WMS Client test suite works, as it shows what one can and cannot do with the Secure Client test suite.

The WMS Client test suite uses TEAM Engine’s MonitorServlet class to allocate a new Uniform Resource Locator (URL) endpoint for tests, and assign a remote URL as the actual recipient of requests. This URL endpoint is then given to the test user, which they input to their client. The WMS client test suite uses a JavaScript-based program to set up the test session on what typically is the test session configuration page in TEAM Engine. As the client interacts with the proxy endpoint, details of the interactions are stored in the test session directory. The WMS Client test suite uses TEAM Engine’s ability to create custom endpoints (using TEAM Engine SPI JAX-RS (Java API extension for RESTful Web services) ApplicationComponents, a separate Application Programming Interface (API) from TEAM Engine Web’s MonitorServlet) to allow the JavaScript program on the test configuration page to query the status of the test, by retrieving the metadata that the client had sent to the proxy endpoint. This metadata is then analyzed by the JavaScript program and displayed to the user as a checklist of conformance and requirements according to the standard. When the test user is finished executing the checklist, they will submit the test form (what usually would be the "start test" action in TEAM Engine), and the JavaScript program will send the client test results to TEAM Engine for conversion into a test report.

There are some downsides to this testing method. Using TEAM Engine APIs in the test suite requires the test suite to run under TEAM Engine, and cannot run as a standalone Java ARchive (JAR) file. Running as a standalone JAR is useful for some test users who may want to automate testing using an interface-less automation framework, or for those who do not want to set up a Java Servlet Container such as Apache Tomcat. TEAM Engine does have a command-line console interface that does not require a servlet container to use, however, the WMS Client test suite must be run from a web browser and from a servlet container application for the JavaScript test program to run. The test suite also does not rely on TestNG (Test Next Generation) and server-side conformance classes, which means the JavaScript program is responsible for the conversion to TEAM Engine reports.

On the upside, the web-based checklist automatically updates as the test user runs their client. This provides excellent visual feedback to the test user that they are correctly using the client. As a test suite developer, updating and debugging a JavaScript-based program would have a faster development loop than Java, which requires recompilation between development tests; a JavaScript program can be updated by editing the Compliance Test Language (CTL) script directly or debugged in a web browser. Using a proxy to actual WMS implementations means the test suite does not need to include its own WMS implementations and can rely on those external implementations for providing data.

For the Secure Client test suite, the testbed sought to also provide dynamic URL endpoints for secure clients to use when testing their conformance. The testbed also tried to provide a clear user interface that shows the test user what to expect and what to do with their client when executing the test suite. The test suite should also included server-side conformance classes implemented with TestNG, as these are easier to document, and the documentation is automatically generated for test users and test suite developers.

Using these lessons from the WMS Client test suite, as well as the requirements and use cases for the Secure Client test suite, the following workflow was proposed:

The source code for the secure client test suite is divided into multiple packages. The main package is for running TestNG, the test executor framework used for OGC executable test suites. This portion of the code is responsible for receiving the test run properties from either a file when executed by a tester via a JAR, an Integrated Development Environment (IDE) or from a dynamically generated file from TEAM Engine. This TestNG Controller class will then validate the properties for correctness and set up an embedded test HTTP web server.

The embedded web server module will start a server with the parameters from the test run properties. These include server details such as the address, port, and hostname, as well as what kind of server to emulate and what secure service annotations to apply to its capability document. This server runs as a unified HTTP/HTTPS service supporting both protocols on the same port, as the test suite must check if a secure client is using HTTPS or not to make requests.

Currently, the test suite supports emulating WMS 1.1.1, WMS 1.3.0, and Web Processing Service (WPS) 2.0 (for OWS Common); additional types could be emulated by updating the test suite with sub-classes that can respond to "Get Capabilities" requests. Once the embedded server is started, a URL for that HTTP server is sent to the test user, either through the command line (for JAR/IDE) or on the web interface (for TEAM Engine).

The test user then connects their secure client to the test endpoint URL. For a simple no-authentication test workflow the test server waits to receive one request from the secure client, sends a response, and shuts down the test server. For more complex workflows such as SAML 2.0, the test server will wait for a pre-determined number of requests from the secure client before shutting down. Each request from the client is logged, converted to eXtensible Markup Language (XML), and saved to a temporary directory.

Once the test server has shut down, the TestNG Controller will take over and send the test run properties as well as the temporary file containing the secure client requests to the TestNG conformance class test packages. If there is an issue with setting up the test server such as missing test run properties, inability to bind the server, timeout waiting for secure client requests, and other potential issues will cause the TestNG controller to abort the embedded test server and pass the partial test run properties and results to the test methods.

The TestNG conformance class test methods are grouped in packages according to conformance classes from the OGC Web Services Security Candidate Standard. These methods can validate the secure client request encoding and correctness with passed, failed, and skipped tests being logged in a report. The report is presented to the user at the end of the test run — when ran via JAR/IDE, an XML report is produced, and when running via TEAM Engine, the report is transformed into an HTML report that can be read in the web browser.

Implementation of the Secure Client test suite encountered multiple issues that complicated development. Understanding these issues, what caused them, and the potential solutions are useful for developers of not only the Secure Client test suite, but can be re-used in the development of other future client test suites.

The main development issues are as follows.

When developing the Identity Provider in AWS, two issues surfaced. The first issue was that the API Gateway decodes any URL encoding before calling a Lambda. This provides a challenge with the XML Signature passed on the URL of the redirect to the Identity Provider. The Identity Provider needs to use URL encoding to encode the signed content before validating the signature. However, if the Service Provider encodes the content slightly differently, the Signature will not validate. You could calculate the signature on the base64 encoding before encoding for the URL in the Service Provider. This would allow the Identity Provider to verify the signature without re-encoding the content; however, this isn’t how SAML is supposed to work.

The second issue encountered is that API Gateway remaps some common HTTP headers as Amazon does not want to implement anything around these headers. However, this prevents anyone else from implementing anything around these headers. The header that causes grief with our scenario is the WWW-Authenticate and Authorization headers for Basic Authentication support. To overcome this issue, the Identity provider needs to pass a custom WWW-Authenticate header (X-WWW-Authenticate) back to the browser. To prevent non-standard issues, the design uses a CloudFront Lambda@Edge to remap the custom header back to the standard WWW-Authenticate header. The Identity provider looks for the response back from the browser in a header that amazon remapped it to (x-amzn-remapped-Authentication). From the consumer side, this appears to be a standards-based exchange. From the Identity providers viewpoint, this is a custom non-standard exchange.

One more consideration to keep in mind for testing and debugging, the Lambda@Edge writes log messages to the data center closest to the edge server. Due to demand and load balancing, logs can go to different data centers. Adding an X-Lambda-Region header with the region where the Edge Lambda is run can help to troubleshoot efforts.

The User Agent illustrated in Figure 2 is the ArcGIS Desktop Plugin. The plugin calls the GetCapabilities method on the Web Map Service (WMS). The capabilities document includes a security description that informs the plugin that it will be using a SAML Browser SSO process flow. This means the plugin must support HTTP redirection and HTML document processing as part of the workflow involves user interaction with a web page. The plugin also needs to support cookies as the assertion for subsequent calls will be stored in a cookie. The plugin then makes a request to the WMS for the full capabilities document by calling the GetCapabilities method listed in the public version. The WMS, acting as the Service Provider in Figure 2, then sends back a redirect to the plugin which then accesses the Discovery Service.

The Discovery Service provides a choice to the plugin as an HTML document for how it wants to authenticate. The plugin allows the user of the plugin to make a choice and submits it to the Discovery Service. Then, the Discovery Service sends a redirect back to the Plugin for the Identity Provider associated with the choice. The plugin then goes to the Identity provider which provides a means to elicit values for the authentication. The plugin allows the user to fill in the information and sends it to the Identity Provider. The Identity Provider responds back with a redirect to the Service Provider which then validates the assertion provided by the Identity Provider and sends a redirect back to the plugin for the target resource (GetCapabilities method). The plugin calls the GetCapabilities method and gets the full capabilities document back.

At this point the plugin follows the same flow for making a GetMap request; but this time, the plugin will be able to pass an assertion that the Service Provider (e.g. WMS) understands and will service the resource without doing the rest of the process flow.

The secure client executable test suite must verify if a secure client uses HTTPS on the connection, as HTTPS is mandatory. To do this, the test suite must be able to mark a request from the secure client as having HTTPS or not. The test suite server does this by using port unification, which allows the test suite to act as an HTTP or HTTPS server depending on the client’s opening message. As the client sends the first message, the server can "peek" at the headers to see if it is a TLS header for HTTPS. If the headers are missing, then the client can be assumed to be connecting with plain HTTP. The test suite will then respond with its capabilities document even if HTTPS is not available; however, the lack of HTTPS will be marked in the metadata that is sent to the test methods. By still accepting HTTP queries, the test suite can evaluate clients lacking HTTPS on other conformance test methods while marking the HTTPS test method as a failure.

For a secure client, they must also support partial capabilities as defined in the standard. This, however, cannot be evaluated from the test suite (i.e. server side), as the parsing and interpretation of the partial capabilities occur entirely in the client. The test user must evaluate that their secure client was able to parse the partial capabilities document for the full capabilities document URL, get the full capabilities and pass any application security controls, and parse the "Content" section from the full capabilities document.

The report generated by the test suite contains a note for the test user that they must evaluate the partial capabilities support manually, as well as provide a checklist and reference to the standard. As the test suite code does not assess it, then categorizes it as a "Skipped" test instead of a "Passed" or "Failed" test. This is provided as a convenience to the test user, reminding them of additional steps necessary to evaluate their client’s conformance.

There are three conformance classes (WMS 1.1.1, WMS 1.3.0, and OWS Common) that each has their own requirements class. In addition to these classes, there are requirement classes for different security annotations adding more constraints to the capabilities presented to secure clients. Some of these classes are related to protocol behavior and supporting HTTP features, and the rest of the classes are for specification of authentication and authorization.

For OGC Testbed 14, the secure client test suite implemented the main HTTP requirement classes and one of the authorization requirement classes. "HTTPS" was provided using TLS on the test suite embedded server. This embedded test server only supports "HTTP/1.1", one of the classes; neither "HTTP/1.0" or "HTTP/2" are enabled. Advertising for "W3C CORS (Cross-origin resource sharing)" is also enabled, as clients that use CORS (particularly web browsers) require that it is enabled to query the service. "HTTP Exceptions" will use standard HTTP status codes and messages with the exceptions returned by the service, linking the exceptions to the category of error (HTTP 4xx — client error, HTTP 5xx — server error).

For authentication, only "SAML2" is supported. This provides the SAML2 Web Browser SSO Profile for secure clients, as that profile had been built for the secure client as part of this testbed. Activating this requirement class in a test session will affect the annotations and the behavior of the test suite. The secure client will be expected to make multiple requests instead of only one.

The other authentication requirement classes were not added to the test suite as they require significantly more development and testing than the other classes. This includes coding for building additional responses, communication with external identity services, and supporting the three main conformance classes. Testing will also require secure clients that support these authentication methods, and no secure clients with support were available for this testbed. Some of the authentication requirement classes share some functionality with the SAML2 workflow, meaning code and procedures can be re-used to build support in the future.

The details for the test suite (ets-security-client10) are accessible from its GitHub Repository and a demonstration video is available on OGC portal and its YouTube channel.

This executable test suite (ETS) verifies the conformance of the secure client behavior as the Implementation Under Test (IUT) with respect to the following specification(s):

Conformance testing for the OGC Web Services Security 1.0 standard is a kind of "black box" testing that examines the externally visible characteristics or behaviors of the IUT while disregarding any implementation details.

Several conformance classes are defined in the principal specification; the ones listed below are covered by this test suite:

The following snapshots (Figure 3) show two snapshots of the TEAM Engin while running the test.

Visit the project documentation website for more information, including the API documentation.