Reusable acceptance criteria and test cases for accessibility

Reusable acceptance criteria
and test cases for accessibility
Sarah Pulis
Director Accessibility Services
Tweet at me: @sarahtp
creating an inclusive digital world
intopia.digital

Setting the scene
Accessibility remediation of select pages on a
transactional portal
Very little native semantic HTML
One accessibility specialist supporting a team
of 8 developers and 3 QA
Team had limited knowledge of accessibility,
WCAG 2.0 and assistive technologies
Aggressive timelines

The QA team was required to record each test
that passed/failed for each test case

Establishing your test objectives
For each release, the product must meet WCAG
2.0 to a particular level, and work as expected
with a defined list of technology combinations

Types of acceptance criteria
Page-level acceptance criteria
Component-level acceptance criteria
(generic or custom)

Must only be tested once on a page, such as page
title or can only be tested in context of the page,
such as reading order

Component-level acceptance criteria
Widgets or common design patterns that can be
tested discretely, such as form inputs, show/hide
content sections or modal and non-modal dialogs

Page title
Page level
Headings
Page level
Text field
Component level
Button
Component level
Group of input fields
Multiple component level
Simple example of page-level
versus component-level criteria
Language of page
Page level
Content order
Page level

Content order (1.3.2 - A)
Focus order (2.4.3 – A)
Sensory characteristics (1.3.3 – A)
Use of colour (1.4.1 – A)
Contrast (1.4.3 – AA)
Resize text (1.4.4 – AA)
Headings (1.3.1 – A; 2.4.6 – AA)
Language of page (3.1.1 – A)
Consistent navigation (3.2.3 – AA)
Consistent identification (3.2.4 – AA)
No keyboard trap (2.1.2 – A)
Timing adjustable (2.1.1 – A)
Seizures (2.3.1 – A)
Bypass blocks (2.4.1 – A)
Page titled (2.4.2 – A)
Link purpose (2.4.4 – A)
Multiple ways (2.4.5 – A)
Parsing (4.1.1 – A)

Simple component-level acceptance criteria
Images
decorative images
meaningful images
simple image
complex image
images of text
Multimedia
audio
video
audio-control
Forms
input and select fields
buttons
CAPTCHAs
On focus
On input
Errors
Tables …

Complex component-level acceptance criteria
Generic complex components
Carousels
Accordions
Modal and non-modal dialogs
Menus
Sliders
Dropdown
Custom complex components
Components that are specific to
the project

Sample acceptance criteria and test cases

Acceptance criteria
Acceptance criteria are often expressed using
behaviour-driven development (BDD) which
uses a given-when-then template to define
scenarios:
Given some initial context,
When an event occurs,
Then ensure some outcomes.

Acceptance criteria for page title
Given that I am on a page
When I read the page title
The page title identifies the content of the page (2.4.2 Page
Titled – A)
The title is different from other pages in the site or app, and
adequately distinguishes the page from other pages (2.4.2
Page Titled – A)

Test process for code inspection
Examine the source code of the HTML or XHTML document
and check that a non-empty <title> element appears in the
<head> section.
Check that the page title adequately and briefly describes the
content of the page.
Check that the title is different from other pages on the site or
app, and adequately distinguishes the page from other pages.

Test process for screen readers
Input Expected result
1. Navigate to a new page
The screen reader announces the page
title
But we were working on a single-page app…

Acceptance criteria for checkboxes (1)
Given that I am on a page with a checkbox or checkboxes
When I press the TAB key to move focus to a checkbox
I see that focus is on the checkbox (2.1.1 Keyboard; 2.4.7 Focus Visible)
And the focus indicator does not rely solely on a colour change (1.4.1 Use
of Color)
When I use a screen reader AND use the keyboard to focus on a checkbox
I hear that I am on a checkbox (4.1.2 Name, Role, Value)
I hear a descriptive label for the checkbox (1.3.1 Info and Relationship,
3.3.2 Labels or Instructions)
And I hear that the checkbox is unselected (4.1.2 Name, Role, Value)

Given that a checkbox has focus
When I press the Space key to toggle the checkbox state
I see that the checkbox toggles between checked and unchecked (2.1.1
Keyboard; 4.1.2 Name, Role, Value)
When I use a screen reader AND I press the Space key to toggle the
checkbox state
I hear that the checkbox toggles between checked and unchecked
(2.1.1 Keyboard; 4.1.2 Name, Role, Value)
And I MAY hear that I am on a checkbox and the descriptive label for
that checkbox

Only required when there are checkboxes that have a logical grouping
and require an additional label or description for the group (see
example).
Given that there are checkboxes are presented as a logical group with a
group label
When I press the TAB key to move focus to the first checkbox in the
group
I hear the label for the group of checkboxes (1.3.1 Info and
Relationship; 3.3.2 Labels or Instructions)

Test process for code inspection
Check that the role of checkbox is specified using one of
the two HTML role methods (native and ARIA).
Check that the name of the radio button is specified using
one of the name methods (label, title, aria-label, aria-
labelledby).
If a native HTML checkbox was used to specify the role,
then use the native HTML checked state. Otherwise use the
custom ARIA methods for role and state.

Test process for keyboard
Input Expected results
1. Use the TAB key to move focus to a
checkbox
Focus moves to the checkbox.
You can see a clear visual indicator that
shows that the checkbox has focus.
The indicator does not rely on a colour
change.
2. Use the Space key to check/uncheck
the checkbox
The checkbox changes state. (e.g. from
checked to unchecked, or unchecked to
checked).

Test process for NVDA/JAWS
Input Expected result
1. Use the TAB key to move focus to a checkbox
The screen reader announces the role of
checkbox.
The screen reader announces that the correct
state for the checkbox (selected or unselected).
The screen reader announces the associated
label for the checkbox.
[Optional] The screen reader announces a label
for checkboxes that have a logical grouping.
2. Use the Space key to check/uncheck the
checkbox
The screen reader announces the checkbox is
selected or unselected (depending on current
state).
[Optional] The screen reader announces the role
of checkbox and the associated label.

You can add additional test processes for assistive
technologies, such as:
VoiceOver on iOS (touch)
TalkBack on Android (touch)
Dragon Naturally Speaking

Bringing it together
All page-level test cases and relevant
component-level test cases were executed
by the QA team
Accessibility specialist was on hand for any
questions or concerns

Benefits
Provides the detailed test cases
required by the QA team
Provides instructions and expected
outputs for keyboard and screen
reader testing (and others can be
added)
Acceptance criteria can also be used
to define requirements for projects
that embed accessibility up front
Reusable across projects and by
distributed teams
Concerns
Reusable acceptance criteria will not
cover every single case
Initially requires an accessibility
specialist to help create the
acceptance criteria and test cases
Custom test cases require
accessibility knowledge
Does not give guidance on
automated testing

Let’s continue the conversation
Sarah Pulis
Director Accessibility Services
Tweet at me: @sarahtp @intopiadigital
creating an inclusive digital world
intopia.digital

Reusable acceptance criteria and test cases for accessibility

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Reusable acceptance criteria and test cases for accessibility

Similar to Reusable acceptance criteria and test cases for accessibility (20)

More from Intopia

More from Intopia (20)

Recently uploaded

Recently uploaded (20)

Reusable acceptance criteria and test cases for accessibility

Editor's Notes