Accessibility Testing With Pa11y CI

Quick setup instructions to run accessibility tests with Pa11y CI.

Tools

  1. Install Nodejs.
  2. Install Chrome.
  3. Install ChromeDriver. If you use Homebrew, you can install it with brew install chromedriver.
  4. Install the needed pa11y packages with:
npm -g install pa11y-ci pa11y-ci-reporter-html

Config

Save this as .pa11yci:

{
  "defaults": {
    "reporters": ["pa11y-ci-reporter-html"],
    "runners": ["axe", "htmlcs"],
    "includeNotices": false,
    "includeWarnings": true,
    "standard": "WCAG2AAA"
  }
}
  • reporters: Configures the HTML reporter. Alternatively, you can set this to cli.
  • runners: Configures pa11y to run tests with both axe and htmlcs.
  • includeNotices: Set to false here. Most of the notices you would see are just manual checks. It might be worth looking into these suggestions at some point, but you should probably focus on fixing the errors and warnings first.
  • includeWarnings: Set to true. You can set it to false if it is too noisy.
  • standard: Configures pa11y to use the strictest standard WCAG2AAA. You can change it to WCAG2AA or WCAG2A to make it less strict. It may make sense to start with the most lenient standard and work yourself up. This setting is only used by htmlcs.

Running tests by passing a sitemap

If your site publishes an XML sitemap, you can just pass the sitemap URL to pa11y-ci:

pa11y-ci --sitemap https://mathiaspolligkeit.com/sitemap.xml

If your sitemap has a lot of pages, this may take a while. Once finished, you can open the report in your browser.

open pa11y-ci-report/index.html

Running tests by configuring the URLs

If your site does not publish a sitemap, you can alternatively add a list of URLs to the config.

{
  "defaults": {
    "reporters": ["pa11y-ci-reporter-html"],
    "runners": ["axe", "htmlcs"],
    "includeNotices": false,
    "includeWarnings": true,
    "standard": "WCAG2AAA"
  },
  "urls": [
    "https://mathiaspolligkeit.com",
    "https://mathiaspolligkeit.com/dev",
    "http://localhost:1313/dev/accessibility-testing-with-pa11y-ci"
  ]
}

Then you can run:

pa11y-ci

Resources

There are a lot more configuration options you can set. For example, you can configure actions to perform before running the tests, for example to log in. Any configuration options listed in the pa11y readme can be added to the defaults object in the config.

For more information, refer to: