WCAG Testing

Last modified by Lucas Charpentier (Sereza7) on 2025/06/26 11:31

Strategy

Our accessibility goal is defined in our Development Practices.

The XWiki WCAG strategy to fulfill this goal is twofolds:

  • XWiki 15.2+ Using Axe Core to automatically detect accessibility violations against WCAG 2.2 (AA level). This should detect about half of the accessibility issues on tested web pages (see below to see what pages are tested).
  • Manually doing WCAG tests in the same manner as we do manual UI tests. This is needed to fully pass the WCAG 2.2 AA specification.

In addition, developers can use WCAG validation tools in their browsers (such as Axe devTools, see at the end for other resources) to reproduce WCAG problems on their computers easily and make it simpler to visually see the problems.

See the XHTML & CSS Coding Style which lists some best practices to be applied by developers and that are not automatically tested.

Testing

We reuse the XWiki docker-based functional tests to perform WCAG validations. This is done automatically at the end of the constructor of each BasePage PO.

This does not cover all possible states of the interface. However it's enough to cover all the static interface (see the Future section below for improvement ideas).

The results of the validation on each page will not fail any test per-see, but it'll fail the test suite if there are WCAG violations. Reports can be found in target/wcag-reports/*.txt files.

There are two WCAG validation rule types:

  • Failing rules: if a violation is found, the test suite fails (and thus the build). The violations are also output in a target/wcag-reports/wcagFails.txt file.
  • Warning rules: if a violation is found, the test-suite does not fail but a warning is displayed. The violations are also output in a target/wcag-reports/wcagWarnings.txt file.

The strategy is to move all the rules to be failing rules. The idea is to do this once we fix all the violations for a given rule so that no more violations can be added in the future without breaking the build.

XWiki 15.8+  In addition to violations, information about Incomplete tests (in need of additionnal manual testing to validate) are stored in the target/wcag-reports/wcagIncompletes.txt file. For every test suite in the build, aggregated statistics about the test results can now be found in target/wcag-reports/wcagOverview.txt.

Reading through the reports

Testing Sessions

Since WCAG tests are not fully automated (and not full automatable), we have manual testing sessions that we do from time to time:

History

Dutch Web Guidelines

Dutch Web Guidelines testing

WCAG Resources

WCAG validators

The following WCAG testing tools are listed for reference:

Online

Browser based

Manual testing checklist

Get Connected