Accessibility Testing Automation

Accessibility bugs affect real users. Screen readers cannot navigate your page, keyboard users cannot reach buttons, low-vision users cannot read your text. These are not edge cases — over one billion people worldwide have some form of disability. Automated accessibility testing catches the mechanical violations: missing alt text, insufficient color contrast, broken ARIA attributes, missing form labels.

Automation cannot catch everything — it finds roughly 30-50% of accessibility issues. But the issues it catches are the easy ones to fix, and catching them automatically means developers fix them before users encounter them.

Prerequisites

• Node.js installed (v16+)
• A web application with HTML pages
• Basic understanding of HTML semantics

Understanding WCAG

The Web Content Accessibility Guidelines (WCAG) define success criteria at three levels:

• Level A — Minimum accessibility. Missing these creates barriers for many users.
• Level AA — Standard target for most organizations. Required by many regulations.
• Level AAA — Highest level. Not always achievable for all content types.

Common violations automated tools catch:

Violation	WCAG Criterion	Impact
Missing alt text	1.1.1	Screen readers cannot describe images
Low color contrast	1.4.3	Text unreadable for low-vision users
Missing form labels	1.3.1	Screen readers cannot identify form fields
Missing page language	3.1.1	Screen readers use wrong pronunciation
Empty links	2.4.4	Screen readers announce "link" with no context
Missing heading structure	1.3.1	Navigation by headings is broken

axe-core: The Foundation

axe-core is the most widely used accessibility testing engine. Most other tools (including browser extensions, CI tools, and testing libraries) use axe-core under the hood.

Using axe-core Directly

npm install --save-dev axe-core

// axe-check.js
var axe = require("axe-core");
var jsdom = require("jsdom");
var JSDOM = jsdom.JSDOM;
var fs = require("fs");

function checkAccessibility(htmlContent) {
  var dom = new JSDOM(htmlContent, { runScripts: "outside-only" });
  var document = dom.window.document;

  // axe-core needs to run in a document context
  return new Promise(function(resolve) {
    axe.run(document.documentElement, {
      rules: {
        // Focus on WCAG 2.1 AA
        "color-contrast": { enabled: true },
        "label": { enabled: true },
        "image-alt": { enabled: true },
        "html-has-lang": { enabled: true }
      }
    }, function(err, results) {
      if (err) throw err;
      resolve(results);
    });
  });
}

// Check a local HTML file
var html = fs.readFileSync("index.html", "utf8");
checkAccessibility(html).then(function(results) {
  if (results.violations.length > 0) {
    console.log("Found " + results.violations.length + " violations:\n");
    results.violations.forEach(function(violation) {
      console.log("Rule: " + violation.id);
      console.log("Impact: " + violation.impact);
      console.log("Description: " + violation.description);
      console.log("Elements affected: " + violation.nodes.length);
      violation.nodes.forEach(function(node) {
        console.log("  " + node.html);
        console.log("  Fix: " + node.failureSummary);
      });
      console.log("");
    });
  } else {
    console.log("No accessibility violations found");
  }
});

Playwright + axe-core

The most powerful combination for automated accessibility testing. Playwright renders real pages; axe-core analyzes them.

Setup

npm install --save-dev @playwright/test @axe-core/playwright

Writing Accessibility Tests

// accessibility.spec.js
var { test, expect } = require("@playwright/test");
var AxeBuilder = require("@axe-core/playwright").default;

test.describe("Accessibility", function() {
  test("homepage has no violations", async function({ page }) {
    await page.goto("http://localhost:3000");

    var results = await new AxeBuilder({ page }).analyze();

    expect(results.violations).toEqual([]);
  });

  test("articles page has no violations", async function({ page }) {
    await page.goto("http://localhost:3000/articles");

    var results = await new AxeBuilder({ page }).analyze();

    expect(results.violations).toEqual([]);
  });

  test("contact form has no violations", async function({ page }) {
    await page.goto("http://localhost:3000/contact");

    // Test the form in its default state
    var defaultResults = await new AxeBuilder({ page }).analyze();
    expect(defaultResults.violations).toEqual([]);

    // Fill out the form to test dynamic states
    await page.fill("#name", "Test User");
    await page.fill("#email", "invalid-email");
    await page.click("button[type=submit]");

    // Test the form with validation errors showing
    var errorResults = await new AxeBuilder({ page }).analyze();
    expect(errorResults.violations).toEqual([]);
  });
});

Filtering by WCAG Level

test("meets WCAG 2.1 AA", async function({ page }) {
  await page.goto("http://localhost:3000");

  var results = await new AxeBuilder({ page })
    .withTags(["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"])
    .analyze();

  expect(results.violations).toEqual([]);
});

Excluding Known Issues

While fixing violations, exclude specific rules temporarily:

test("homepage accessibility (excluding known issues)", async function({ page }) {
  await page.goto("http://localhost:3000");

  var results = await new AxeBuilder({ page })
    .disableRules(["color-contrast"]) // Will fix in issue #456
    .exclude(".third-party-widget")    // Cannot control third-party markup
    .analyze();

  expect(results.violations).toEqual([]);
});

Testing Specific Components

test("navigation is accessible", async function({ page }) {
  await page.goto("http://localhost:3000");

  var results = await new AxeBuilder({ page })
    .include("nav")
    .analyze();

  expect(results.violations).toEqual([]);
});

test("footer is accessible", async function({ page }) {
  await page.goto("http://localhost:3000");

  var results = await new AxeBuilder({ page })
    .include("footer")
    .analyze();

  expect(results.violations).toEqual([]);
});

pa11y: Command-Line Accessibility Testing

pa11y is a standalone accessibility testing tool that runs from the command line.

Setup

npm install --save-dev pa11y

Basic Usage

npx pa11y http://localhost:3000

Output:

Results for http://localhost:3000

 • Error: <img src="logo.png"> - Images must have alternate text
   WCAG2AA.Principle1.Guideline1_1.1_1_1.H37
   #header > img

 • Error: <input type="email"> - This form field should be labelled
   WCAG2AA.Principle1.Guideline1_3.1_3_1.F68
   #newsletter-signup > input

 • Warning: <p style="color: #999"> - This element has insufficient contrast
   WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail
   .footer-text

3 Errors, 1 Warning

pa11y Configuration File

{
  "defaults": {
    "standard": "WCAG2AA",
    "timeout": 10000,
    "wait": 1000,
    "chromeLaunchConfig": {
      "args": ["--no-sandbox"]
    }
  },
  "urls": [
    "http://localhost:3000",
    "http://localhost:3000/articles",
    "http://localhost:3000/contact",
    {
      "url": "http://localhost:3000/login",
      "actions": [
        "set field #email to [email protected]",
        "set field #password to password123",
        "click element button[type=submit]",
        "wait for url to be http://localhost:3000/dashboard"
      ]
    }
  ]
}

pa11y-ci for Continuous Integration

npm install --save-dev pa11y-ci

{
  "defaults": {
    "standard": "WCAG2AA",
    "timeout": 30000
  },
  "urls": [
    "http://localhost:3000",
    "http://localhost:3000/articles",
    "http://localhost:3000/contact"
  ]
}

{
  "scripts": {
    "test:a11y": "pa11y-ci"
  }
}

# GitHub Actions
- name: Accessibility tests
  run: |
    npm start &
    sleep 5
    npx pa11y-ci

Jest + axe-core for Unit-Level Testing

Test HTML fragments or rendered components:

npm install --save-dev jest-axe

// component.test.js
var { axe, toHaveNoViolations } = require("jest-axe");

expect.extend(toHaveNoViolations);

test("user card has no accessibility violations", function() {
  var html = '<div class="user-card">'
    + '<img src="avatar.jpg" alt="User avatar">'
    + '<h3>Shane Developer</h3>'
    + '<p>Full-stack engineer</p>'
    + '<a href="/profile/shane">View Profile</a>'
    + '</div>';

  return axe(html).then(function(results) {
    expect(results).toHaveNoViolations();
  });
});

test("form without labels fails accessibility", function() {
  var html = '<form>'
    + '<input type="text" name="username">'
    + '<input type="email" name="email">'
    + '<button type="submit">Submit</button>'
    + '</form>';

  return axe(html).then(function(results) {
    expect(results.violations.length).toBeGreaterThan(0);
    expect(results.violations[0].id).toBe("label");
  });
});

Keyboard Navigation Testing

Automated tools miss keyboard accessibility issues. Playwright can test keyboard navigation:

test("can navigate form with keyboard", async function({ page }) {
  await page.goto("http://localhost:3000/contact");

  // Tab to the first field
  await page.keyboard.press("Tab");
  var focused = await page.evaluate(function() {
    return document.activeElement.tagName + "#" + document.activeElement.id;
  });
  expect(focused).toBe("INPUT#name");

  // Tab to the next field
  await page.keyboard.press("Tab");
  focused = await page.evaluate(function() {
    return document.activeElement.tagName + "#" + document.activeElement.id;
  });
  expect(focused).toBe("INPUT#email");

  // Tab to submit button
  await page.keyboard.press("Tab");
  await page.keyboard.press("Tab"); // Skip message textarea
  focused = await page.evaluate(function() {
    return document.activeElement.tagName;
  });
  expect(focused).toBe("BUTTON");
});

test("modal can be closed with Escape key", async function({ page }) {
  await page.goto("http://localhost:3000");

  // Open modal
  await page.click(".open-modal-button");
  await page.waitForSelector(".modal", { state: "visible" });

  // Press Escape
  await page.keyboard.press("Escape");

  // Modal should be hidden
  await expect(page.locator(".modal")).not.toBeVisible();
});

test("skip link moves focus to main content", async function({ page }) {
  await page.goto("http://localhost:3000");

  // Tab to the skip link
  await page.keyboard.press("Tab");

  var skipLink = await page.evaluate(function() {
    return document.activeElement.textContent;
  });
  expect(skipLink).toContain("Skip to content");

  // Activate the skip link
  await page.keyboard.press("Enter");

  // Focus should be on the main content area
  var focusedId = await page.evaluate(function() {
    return document.activeElement.id;
  });
  expect(focusedId).toBe("main-content");
});

Custom Accessibility Test Helpers

// a11y-helpers.js
var AxeBuilder = require("@axe-core/playwright").default;

function checkPageAccessibility(page, options) {
  var builder = new AxeBuilder({ page })
    .withTags(["wcag2a", "wcag2aa"]);

  if (options && options.exclude) {
    options.exclude.forEach(function(selector) {
      builder.exclude(selector);
    });
  }

  if (options && options.disableRules) {
    builder.disableRules(options.disableRules);
  }

  return builder.analyze().then(function(results) {
    return {
      violations: results.violations,
      passes: results.passes.length,
      incomplete: results.incomplete.length,
      summary: formatViolations(results.violations)
    };
  });
}

function formatViolations(violations) {
  if (violations.length === 0) return "No violations found";

  var lines = ["Found " + violations.length + " violation(s):"];

  violations.forEach(function(v) {
    lines.push("");
    lines.push("[" + v.impact.toUpperCase() + "] " + v.id + ": " + v.description);
    lines.push("  Help: " + v.helpUrl);
    v.nodes.forEach(function(node) {
      lines.push("  Element: " + node.html.substring(0, 100));
    });
  });

  return lines.join("\n");
}

function expectNoViolations(results) {
  if (results.violations.length > 0) {
    throw new Error(results.summary);
  }
}

module.exports = {
  checkPageAccessibility: checkPageAccessibility,
  expectNoViolations: expectNoViolations
};

var a11y = require("./a11y-helpers");

test("all pages are accessible", async function({ page }) {
  var pages = ["/", "/articles", "/contact", "/about"];

  for (var i = 0; i < pages.length; i++) {
    await page.goto("http://localhost:3000" + pages[i]);
    var results = await a11y.checkPageAccessibility(page);
    a11y.expectNoViolations(results);
  }
});

CI/CD Integration

GitHub Actions Workflow

name: Accessibility Tests
on: [pull_request]

jobs:
  a11y:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"
      - run: npm ci
      - run: npx playwright install --with-deps chromium

      - name: Start application
        run: npm start &
        env:
          NODE_ENV: test
      - name: Wait for app
        run: npx wait-on http://localhost:3000

      - name: Run accessibility tests
        run: npx playwright test --grep @a11y

      - name: Run pa11y-ci
        run: npx pa11y-ci

      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: a11y-report
          path: test-results/

Pre-Commit Hook

{
  "scripts": {
    "lint:a11y": "pa11y-ci --sitemap http://localhost:3000/sitemap.xml"
  }
}

Common Violations and Fixes

Missing Alt Text

<!-- BAD -->
<img src="hero.jpg">

<!-- GOOD — descriptive alt text -->
<img src="hero.jpg" alt="Developer working at a standing desk with multiple monitors">

<!-- GOOD — decorative image -->
<img src="divider.png" alt="" role="presentation">

Missing Form Labels

<!-- BAD -->
<input type="email" placeholder="Your email">

<!-- GOOD -->
<label for="email">Email address</label>
<input type="email" id="email" placeholder="[email protected]">

<!-- GOOD — visually hidden label -->
<label for="search" class="sr-only">Search articles</label>
<input type="search" id="search" placeholder="Search...">

Insufficient Color Contrast

/* BAD — gray on white fails WCAG AA (ratio 2.8:1) */
.muted-text { color: #999; }

/* GOOD — darker gray passes WCAG AA (ratio 4.6:1) */
.muted-text { color: #767676; }

/* GOOD — even darker for small text (ratio 7:1) */
.small-text { color: #595959; }

Missing Page Language

<!-- BAD -->
<html>

<!-- GOOD -->
<html lang="en">

Empty Links

<!-- BAD -->
<a href="/profile"><i class="icon-user"></i></a>

<!-- GOOD -->
<a href="/profile" aria-label="User profile"><i class="icon-user"></i></a>

<!-- GOOD -->
<a href="/profile"><i class="icon-user" aria-hidden="true"></i> Profile</a>

Common Issues and Troubleshooting

axe-core reports no violations but users report accessibility issues

Automated tools catch 30-50% of accessibility issues. Manual testing is still required:

Fix: Combine automated testing with manual testing. Test with a screen reader (NVDA on Windows, VoiceOver on Mac). Test keyboard navigation manually. Include users with disabilities in usability testing.

Too many violations to fix at once

A legacy application may have hundreds of violations:

Fix: Prioritize by impact level: critical and serious first, then moderate and minor. Fix violations in new code immediately. Dedicate a small percentage of each sprint to fixing existing violations. Use disableRules to suppress known issues with tracked tickets.

Tests pass but pa11y finds different violations

Different tools use different rule sets and versions of axe-core:

Fix: Standardize on one tool for CI gating. Use others for additional insight. Ensure all tools target the same WCAG level (AA is the standard target).

Dynamic content is not tested

Modals, dropdowns, and AJAX-loaded content are not present when the page first loads:

Fix: Write tests that interact with the page — click buttons to open modals, trigger dropdowns, wait for dynamic content to load — then run accessibility checks. Test each state of the UI, not just the initial load.

Best Practices

• Start with automated testing and iterate toward manual testing. Automated tools catch the low-hanging fruit. Add manual screen reader testing and keyboard testing as your accessibility practice matures.
• Test every page, not just the homepage. Forms, error states, modals, and interactive components often have the worst accessibility. Test the pages users actually interact with.
• Fix violations in new code immediately. Do not add to the backlog. If a PR introduces an accessibility violation, fix it before merging. Prevention is cheaper than remediation.
• Include accessibility checks in CI. Run axe-core or pa11y in your pipeline. Fail the build on critical violations. This prevents regressions.
• Test interactive states. A form might be accessible in its default state but inaccessible when showing validation errors. Test all states: empty, filled, error, success, loading.
• Use semantic HTML before ARIA. A <button> is accessible by default. A <div onclick> needs ARIA roles, keyboard handlers, and focus management. Semantic HTML does the heavy lifting.
• Track violations as technical debt. Each suppressed rule or excluded element should have a ticket. Review the list quarterly and reduce it.
• Learn from violations. Each accessibility bug is a learning opportunity. Share common patterns with the team to prevent recurrence.

References

• axe-core
• Playwright Accessibility Testing
• pa11y
• WCAG 2.1 Guidelines
• WebAIM Contrast Checker
• ARIA Authoring Practices