WCAG Testing
Strategy
Our accessibility goal is defined in our Development Practices.
The XWiki WCAG strategy to fulfill this goal is twofolds:
- XWiki 15.2+ Using Axe Core to automatically detect accessibility violations against WCAG 2.2 (AA level). This should detect about half of the accessibility issues on tested web pages (see below to see what pages are tested).
- Manually doing WCAG tests in the same manner as we do manual UI tests. This is needed to fully pass the WCAG 2.2 AA specification.
In addition, developers can use WCAG validation tools in their browsers (such as Axe devTools, see at the end for other resources) to reproduce WCAG problems on their computers easily and make it simpler to visually see the problems.
See the XHTML & CSS Coding Style which lists some best practices to be applied by developers and that are not automatically tested.
Testing
We reuse the XWiki docker-based functional tests to perform WCAG validations. This is done automatically at the end of the constructor of each BasePage PO.
This does not cover all possible states of the interface. However it's enough to cover all the static interface (see the Future section below for improvement ideas).
The results of the validation on each page will not fail any test per-see, but it'll fail the test suite if there are WCAG violations. Reports can be found in target/wcag-reports/*.txt files.
There are two WCAG validation rule types:
- Failing rules: if a violation is found, the test suite fails (and thus the build). The violations are also output in a target/wcag-reports/wcagFails.txt file.
- Warning rules: if a violation is found, the test-suite does not fail but a warning is displayed. The violations are also output in a target/wcag-reports/wcagWarnings.txt file.
The strategy is to move all the rules to be failing rules. The idea is to do this once we fix all the violations for a given rule so that no more violations can be added in the future without breaking the build.
XWiki 15.8+ In addition to violations, information about Incomplete tests (in need of additionnal manual testing to validate) are stored in the target/wcag-reports/wcagIncompletes.txt file. For every test suite in the build, aggregated statistics about the test results can now be found in target/wcag-reports/wcagOverview.txt.
Reading through the reports
Testing Sessions
Since WCAG tests are not fully automated (and not full automatable), we have manual testing sessions that we do from time to time:
- Session for XE 2.1
- Testing session by NGI Accessibility ()
- February 2024 - Dutch Web Guidelines WCAG testing/
- March 2024 - Accessibility audit by Zendis: original (DE) - machine translations ยท
- August 2024 - Testing session of XWiki 16.6.0 by the HAN University of applied sciences in the context of NGI Zero ()
History
- XWiki 15.5+ Improved log traces of WCAG validation
- XWiki 15.2+ XWiki uses Axe Core to validate accessibility
- XWiki <15.2 XWiki was using the Dutch Web Guidelines to validate accessibility
Dutch Web Guidelines
WCAG Resources
WCAG validators
The following WCAG testing tools are listed for reference:
Online
- TAW3 (Tutorial for the Web start version).
- Total Validator
- Wave (ability to paste HTML content or upload a local file)
Browser based
- Axe devTools
- Firefox Web Developer Tool tutorial for Accessibility (especially precise for contrast issues)
- WAVE extension
- SiteImprove Accessibility Checker
- AInspector
- IBM Equal Access Accessibility Checker (2 sets of rules checkable, that can be changed in parameters)
Manual testing checklist
