3.1.  Overview Document

The Compliance Test Program provides the resources, procedures, and policies to certify products for compliance with one or more OGC standards.

The primary purpose of the Compliance Testing Program is to increase systems interoperability while reducing technology risks by providing a process whereby compliance with OGC standards can be tested. The Compliance Testing Program provides confidence to technology vendors and buyers. Vendors feel confident that they are providing a product compliant with OGC standards, which will be easier to integrate and easier to market. Buyers fill confident that a compliant product will work with another compliant product based on the same OGC standard, regardless of which company developed the product.

3.2.  Changes to this document

Any changes to this document are discussed at the OGC CITE Steering Committee (SC). Main changes are discussed at the Planning Committee (PC). Ultimately the Technical Committee (TC) votes for the release and approval of this document. A change request can also be submitted against this document via the OGC web site at http://ogc.standardstracker.org.

4.1.  CITE Subcommittee

The CITE Subcommittee (CITE SC) is a committee of the OGC TC consisting of OGC Members interested in the Compliance Testing Program. CITE SC has the following responsibilities:

• Guides the direction of the Compliance Testing Program.

• Reviews formal motions brought from OGC Members and OGC Staff.

• Submits formal motions and recommends approval by the TC.

• Reviews and approves roadmap.

4.2.  Director of Interoperability Certification

The Director of interoperability Certification currently oversees the Compliance Testing Program and acts as the Compliance Testing Coordinator.

4.3.  The Compliance Testing Coordinator

The Compliance Testing Coordinator (CTC) is the OGC Staff member responsible for coordinating and managing the operation of the Compliance Testing Program. The CTC has the following responsibilities:

• Propose candidate guidelines, policies, and procedures to the CITE SC.

• Coordinate the developing and maintenance of Compliance Test Packages.

• Coordinate the developing and maintenance of the Compliance Testing Engine.

• Propose candidate Compliance Test Packages to the CITE SC.

• Maintain guidelines and documentation of the Program.

• Select Reference Implementations (RI) for a Compliance Test Package and coordinate with the Reference Implementations developers for proper maintenance.

• Coordinate the Compliance Testing Evaluation of a Candidate Product.

• Manage and execute the interpretation and clarification process in collaboration with Standards Working Groups and Revision Working Groups.

• Manage and execute the dispute and challenge resolution process in collaboration with the
  appropriate working groups and OGC Member discussion forums.

4.4.  The CITE Subcommittee, Technical and Planning Committees (PC)

With respect to compliance testing, the SC, TC and PC have the following responsibilities:

• Review and approve compliance testing guidelines, policies, and procedures.

• Review and select Compliance Test Packages including adaptations and provide an analysis of the adaptations to the CTC.

• Approve the release of Compliance Test Package versions.

4.5.  OGC Compliance Team

The OGC Compliance Team is composed of OGC staff supporting this program. It includes the Compliance Testing Coordinator and other OGC staff.

For compliance testing, the OGC Staff has the following responsibilities:

• Support the Compliance Testing Coordinator related to the Compliance Testing Program.

• Administer the certification process including invoice processing and fees.

.

5.1.  Description of the Compliance Testing Evaluation Procedure

The Compliance Testing Evaluation Procedure is a set of steps used to evaluate a software product for proper implementation of an OGC standard. OGC has setup a web site providing an online procedure for most of the tests; however, there are some tests that are not available via the web site. The tests not available as an online test (e.g. Simple Feature related tests) can be found here:

http://www.ogc.org/compliance/downloads

The online test procedure is detailed in Figure 1. A box indicates a process. A red box indicates a process for which a Candidate Organization (CO) is responsible. A blue box indicates a process for which the Compliance Team (CT) is responsible.

In August of 2013, OGC made available a compliance submission website simplifying the submission process and the need to submit forms and results manually. The forms are available as online web forms and the web site links with the testing website facility. Link: http://www.ogc.org/resource/products/registration

Figure 1

Figure 1 – Flow chart for the Compliance Testing Evaluation Procedure

5.2.  Policy on Previously Approved Products and Future Releases

Compliance will be determined once for a product version with respect to a particular Implementation Standard Version, and requires the successful completion of the Testing Procedure. Compliance of future releases of the same product requires the successful completion of the Testing Procedure.

5.3.  Compliance Determination

Candidate Products that pass all the tests in a Compliance Test Package, and that OGC has reviewed and certified as having passed those tests, are considered compliant with that Implementation Standard version. This is the sole criterion for determining the compliance of a Candidate Product.

5.4.  Test Records

The CTC will maintain a file of records pertaining to testing, hereafter referred to as the OGC Testing Record. Records pertaining to a particular organization will be available for examination by that organization upon request. Records maintained in the OGC Testing Record will be maintained for a minimum of three (3) years.

OGC will make a good faith effort to retain the following documents in the OGC Test Record:

• Abstract Test Suites,

• Executable Test Suites,

• Test Summary Reports (and all attachments),

• Test Failure Notification Letters,

• Compliance Certificates,

• Trademark License Agreements,

• Compliant Product Records, and

• Any other correspondence of importance related to the OGC Compliance Testing Program.

5.5.  Certificate Issuance

A Compliance Certificate (Appendix F) is issued for tested software products that successfully pass the Compliance Testing Evaluation Procedure.

The holder of such a certificate is entitled to use the ”Certified OGC Compliant” mark (Figure 2) according to terms and conditions set forth in the Trademark License Agreement (Appendix E) and after the applicable Trademark Licensing Fee has been paid (Appendix G). The Trademark Licensing Fee shall be paid annually in order to maintain a valid certificate.

Figure 2

Figure 2 – Certified OGC Compliant Mark

The Compliance team will create an OGC Testing Record with a copy of the Compliance Certificate and the supporting Test Summary Report. The Compliance Team will add any product that receives a Compliance Certificate to the Compliant Products Record (Appendix F). Products that are added to the Compliant Products Record will be removed three (3) months after failure to pay the annual trademark-licensing fee, or by written request of the Candidate Organization. Relevant information from the Compliant Products Record will be maintained on the OGC public web site (http://www.ogc.org/) for every product that receives a Compliance Certificate.

In order to avoid confusion in the place, OGC reserves the right to terminate the right of a Candidate Organization to use OGC trademarks to indicate successful Compliance testing at such time as OGC deems consumer confusion likely. In the event that this occurs, OGC shall give advance notice to the extent possible, and transitional steps will be taken, to minimize any inconvenience to vendors of Candidate Products.

4.6. Live Logos

OGC provides a link to live logos for organizations that have successfully passed the Compliance Testing Evaluation Procedure. Every product has its own link where an image is automatically generated. An example is shown in Figure 3. The image contains the company name, the product name and the standards for which the product has been certified with. It also contains the date of validity. The logo links directly to the OGC public record that contains more information about the company.

Figure 3

Figure 3 – Live Logo

6.1.  Parties Who May Develop Compliance Test Packages

Candidate Compliance Test Packages may be developed by any of:

• OGC Members

• The CTC and/or other OGC Staff

• OGC Testing Contractors (under the direction of the CTC)

Candidate Compliance Test Packages must be submitted to the CTC for review and (potential) adoption.

6.2.  Compliance Test Package Maintenance and Criteria for Review

In general, OGC testing staff will maintain CTP under the direction of the CTC. This maintenance responsibility may be delegated to an OGC member or OGC Staff by the CTC. Compliance Test Package maintenance is called for when a new Implementation Standard version is issued by OGC, when the dispute resolution process mandates such a change (see Clause 9), or when the CTC, in his/her sole discretion finds issues that require maintenance.

In any case, the CTC is responsible for the review of minor revisions to the Compliance Test Package. The resulting Compliance Test Package revision will then be made available to the OGC membership via the CITE SC mailing list for a minimum of five business days. The criteria for the CTC review and TC approval is that:

• The Compliance Test Package revision is complete with respect to the Compliance Testing Guideline for that particular Implementation Standard version.

• Consistency with the wording and intent of the particular Implementation Standard version.

• Consistency with OGC policy and practice.

• Versions of compliance test packages are maintained in the configuration management system and recorded with the compliance test execution.

Delivery, testing, configuration control of compliance test packages is described in the CITE Configuration Management Procedures.

Major revisions to a Compliance Test Package shall go through the same process as a new Compliance Test Package, as described in Clause 6, Compliance Test Package Development.

A revision is considered major by the judgment of the CTC. However, any revision resulting in the change of greater than 20 percent of executable tests must be considered a major revision.

6.3.  Compliance Test Package Availability

Compliance Test Packages will be provided in electronic form, when they are available, via OGC’s public web site. The Compliance Test Package will be made available along with full documentation of the OGC Compliance Testing Program (this document) and Compliance Testing Guidelines.

6.4.  Compliance Test Package Submitters

Compliance Test Package submitters must provide either 1) unrestricted ownership of the Compliance Test Package to OGC, or 2) an agreement to grant OGC a non-exclusive, royalty-free, paid-up, unrestricted worldwide license to copy and distribute their submission to the OGC membership, and, if adopted by OGC, the unrestricted right to modify, enhance, and make derivative works from the materials and to copy and distribute the materials, modifications, enhancements, and derivative works both inside and outside of the OGC membership. Materials include software source code, documentation in multimedia or text formats, and in other resource part of the test package or required to perform the testing.

6.5.  Beta Testing impact on Certificates

These first three implementations (can be reference implementations or not) to pass 100% of the compliance procedure, while the test is in beta, will have their Trademark License Fee waived through the annual licensing period for the product.

7.1.  Definition and Determination

A fully functional, licensed copy of a tested, branded software that has passes the test for an associated conformance class in a version of an Implementation Standard and that is free and publicly available for testing via a web service or download. The Reference Implementation does not need to pass all the conformance classes within the standard. In most of the cases, the Reference Implementation will pass at least the core and possibly some number of extension conformance classes. Multiple reference implementations can exist for an associated version of an Implementation Standard. The coordinator after reviewing the results and checking the public interface of the software will determine if the implementation can be a reference implementation.

OGC will make its best effort to host Reference Implementations on an OGC server to help others in the community to develop compliant implementations.

7.2.  Service Level Requirements

If the Reference Implementation is not hosted by OGC, it is expected the following from the organization providing the Reference Implementation

The Reference Implementation should

• be available in a reliable web server with 99% uptime.

• pass the test for every operational release of OGC Testing infrastructure (e.g. TEAM Engine)

If any of the aforementioned conditions fail the CITE coordinator can remove the Reference Implementation label of an existing implementation.

7.3.  Fee Incentives for Reference Implementations

OGC will provide an incentive to the first two Reference Implementations that pass the test related to a conformance class within a version of an Implementation Standard. OGC will waive the license fee every year as long as the Reference Implementation passes again the conformance class test successfully.

9.1.  Dispute Procedure

A “Compliance Issue” is defined as a problem identified from third party review or experience with a product after testing is completed and the product is available in the marketplace. The submitter of the issue should send an email to compliance@ogc.org summarizing the problem.

The receipt of a Compliance Issue by CTC will set the following procedure in motion:

1. The CTC will coordinate that the issue is captured in the Compliance Issue a tracking system.

2. The CTC will provide clarification to the issue, if possible. If not, the CTC will suggest to use the CITE forum.

3. The CTC will notify the disputing organization of the resolution and record the dispute as resolved, closing the issue in the issue tracker. The disputing organization may appeal the decision by submitting another Compliance issue.

4. If issue submitter disagrees with the decision of the CTC, then CTC will forward the Compliance Issue to the OGC Review Board (CITE SC) for resolution along with proposed resolution alternatives and a recommendation. The issue will be opened again in the issue tracker.

5. The CTC will then notify the disputing agreement the decision of the OGC Review Board (CITE SC). And will follows step 3.

9.2.  Adaptation of Compliance Test Packages

Organizations may have to adapt Compliance Test Packages to work with their products. OGC will provide guidance regarding allowable adaptations. In any case, OGC testing staff, in consultation with and under the approval authority of the CTC, must review the adaptation. The OGC testing staff review and CTC approval is to ensure that the original intent and functionality of the test suite is maintained. OGC testing staff must also document the adaptation, by inclusion of the adapted test suite, in the Compliance Test Summary Report (Appendix A). Candidate Organizations must submit adapted Compliance Test Packages at the time they submit a Self-Test Summary Report.

9.3.  Interpretation and Clarification of Standards

Issues regarding the interpretation or clarification of Implementation Standards are the responsibility of the applicable Standards Working Group (SWG) of the OGC Technical Committee (specifically, the SWG that corresponds to the Implementation Standard in question). If no SWG is currently active, then the TC Chair will form an ad hoc committee (comprised of the last standing SWG membership, if possible) to process the interpretation/clarification. The TC Chair can also decide to submit the issue to the OGC Review Board.