Testing a whole site against WCAG, and standing behind the result

WCAG-EM is independent of any tool, and says that most accessibility checks are not fully automatable. No single way of testing covers every criterion, so a test of a page uses three:

Method	What it settles	What it leaves open
Automated checks	Rules a tool can decide from the code, on many pages at once: an image with no alt attribute, the contrast of text it can compute, a field with no name	Whether the words are right, and anything that needs a person to use the page
Inspection	The code and what it gives assistive technology: names, roles, the heading outline, focus order, contrast measured with a colour picker	Whether the screen readers in the baseline actually announce and operate it
Assistive technology	Whether the content works in the baseline combinations, which is what conformance requirement 4 is about	Only as much as the tester's own skill with each one shows

What tools find and miss is on automated checks, and what makes a technique accessibility supported is on whether a technique works in the screen readers people use.

The last row has two limits of its own. WCAG-EM assumes a tester with a solid understanding of assistive technologies and of how people with different disabilities use digital products, and allows a team to bring that expertise together when one person doesn't have all of it. A tester who doesn't know a screen reader's reading and typing modes may report a failure that comes from pressing the wrong key. W3C's page on involving users in web projects notes that a very advanced user may know uncommon work-arounds that an average user could not manage, and a fluent tester can get past a barrier the same way without noticing it.

A magnifier's own settings change what the tester sees. ZoomText's focus enhancements draw its own marker on the control that has focus, and its colour enhancements adjust colour, contrast and brightness. With them on, a page that draws no focus indicator still shows one, and contrast judged by eye on the magnified screen is not the page's contrast. Check 2.4.7 Focus Visible and 1.4.3 Contrast (Minimum) with those enhancements off or with no magnifier, then use the magnifier for what it is for, following focus and reading at high zoom. A page with no focus indicator is in missing focus indicators on custom buttons.

WCAG-EM says which pages to test and what to report, and leaves the test of each criterion to the tester: W3C's techniques may be used but are not required. A test process fills that gap. The Trusted Tester Section 508 Conformance Test Process for Web, version 5.1.3 (DHS, April 2024), is one, and section508.gov describes it as a manual test approach that gives repeatable results. As of this page's date 5.1.3 is the current version; check section508.gov for a newer one.

• What it tests: the 41 web requirements of Section 508, which are 38 WCAG 2.0 level A and AA success criteria and 3 Section 508 requirements, through 63 test conditions.
• How a condition is decided: each test condition is a statement that is true or false, with fixed steps. The result is Pass, Fail, Does Not Apply, or Not Tested, which is allowed only where the tools can't do the test. Every condition needs a result before the test is complete.
• Tools: ANDI (Accessible Name & Description Inspector), a free bookmarklet that shows what a page gives assistive technology, and the Colour Contrast Analyser, or Accessibility Insights for contrast. The process calls itself essentially a code inspection for accessibility properties.
• Order: it recommends Test 1, conforming alternate versions, first; then auto-playing content, flashing, and keyboard access and focus, which cover the criteria non-interference applies to the whole page. After Test 1, the tester may take the tests in any order.
• What it leaves out: which pages to test. It asks for the scope of testing to be stated, and doesn't say how to choose it.

Two of its facts matter for a WCAG 2.2 target. It tests WCAG 2.0, because Section 508 adopts WCAG 2.0, so the criteria added in 2.1 and 2.2, such as 1.4.10 Reflow and 2.5.8 Target Size (Minimum), need tests of their own. And it records 4.1.1 Parsing as Not Tested, a criterion WCAG 2.2 no longer has. Used together, WCAG-EM chooses the pages and Trusted Tester tests the WCAG 2.0 criteria on each.

A finding goes under the criterion whose test it fails, not the one nearest in topic. Three criteria sit close together on every form, and Trusted Tester gives each its own test condition:

Criterion	What it asks of a field	Trusted Tester
3.3.2 Labels or Instructions (A)	A label or instruction is there on screen, and still visible while the field has focus	5.A, labels provided
2.4.6 Headings and Labels (AA)	The label's words say what to enter	5.B, labels descriptive
1.3.1 Info and Relationships (A)	The label is tied to the field in the code	5.C, programmatic label

So the same form can produce three different findings:

• A "Postcode" label whose for points at the wrong id is a 1.3.1 failure: the words are there and they are right, but they label nothing. Form fields with missing or mismatched labels shows it, and the placeholder that vanishes once someone types, which is 3.3.2.
• "Name" on a gift form about two people fails 2.4.6 and only 2.4.6. The label is tied to its field and stays on screen, so the form passes 1.3.1 and 3.3.2; what fails is the words. That is vague form labels that don't say what to enter.
• Headings follow the same split. Trusted Tester tests a heading's words under 2.4.6 (10.A) and its markup and level under 1.3.1 (10.B and 10.C). In headings that don't match the page's sections, section titles styled as div elements fail 1.3.1, and a heading that says only "More" fails 2.4.6.

When one defect fails two tests, as the empty h2 in that guide fails both, list it under each and say it is one fix. Filing every form issue under 1.3.1, or the same defect under all three, makes a report count failures that aren't there and sends developers to the wrong fix.

A failure or a best practice. A failure is content that doesn't meet a success criterion at the target level. Anything else in a report is a best practice and is labelled as one: a criterion above the target, an advisory technique, a usability problem. Form field help only on a help page or on hover is a real barrier, but its criterion, 3.3.5 Help, is level AAA, so under an AA target it goes in the report as a best practice. WCAG-EM notes that evaluating beyond the target can help plan improvements, and Trusted Tester lets problems outside Section 508, such as a link to the wrong site, go in a comment rather than count as a failure. A finding is not a failure because the page doesn't use a W3C technique either: WCAG-EM doesn't require any particular technique.

WCAG-EM's Step 5.1 lists what a report includes, and Trusted Tester asks every conformance report built from it to identify its test process, its scope and its test environment:

Item	What to write	Where it comes from
Date	The completion date or the period of testing	WCAG-EM 5.1
Who	The evaluator and the client who asked for it	WCAG-EM 5.1
Scope	The pages in and out, the target level, the accessibility support baseline	WCAG-EM 5.1, Trusted Tester
Technologies	The technologies relied upon	WCAG-EM 5.1
Sample	The structured pages, the random pages and how they were drawn, each complete process	WCAG-EM 5.1
Method	The methodology and test process, with versions	WCAG-EM 5.1 (optional), Trusted Tester (required)
Tools and versions	Testing tools, browsers and add-ons, assistive technology, operating system	WCAG-EM 5.2 (optional), Trusted Tester (required)
Outcomes	At least one example of each criterion not met, with issues that repeat marked as such	WCAG-EM 5.1

For each failure, WCAG-EM notes that a clear description, steps to reproduce, severity and screenshots help teams fix it, and Trusted Tester asks for where the failure is and, where it can, what caused it. EaseWeb's conformance record is one report built from these items.

An evaluation statement is a different thing from a report. WCAG-EM's Step 5.3 allows one only when every required step was done, every page in the sample meets the target, and the owner commits to keeping it accurate. Even then it is a statement about an evaluation, not a conformance claim for the site, because conformance is defined for each web page: what it takes for a page to conform to WCAG covers what a claim needs.