Skip to content
English
On this page

Accessibility Testing

“Accessibility - Essential for some, useful for all” —— W3C WAI

The web is an essential aspect of our lives in so many ways — we use it to purchase everyday commodities and deliver them to our doorsteps; we learn so much from browsing the web every day. I can’t imagine surviving the Coronavirus pandemic without the connectivity, productivity, and information the web provides. Our lives have become entwined with the web, as a matter of fact. Making such a vital commodity available to all users with challenges is termed ‘web accessibility.’ For instance, enabling a visually challenged person, elderlies, temporary disabled (with a fracture or while driving), people with literary gaps, and so on, to access the web is making the web accessible. It is a subset of ‘usability’ in web development terms. It is a subset of inclusivity under humanitarian grounds. Although accessibility is targeted to enable people with challenges to avail the services of the web, it is just not making their lives easier but enhancing everyone’s lives. I love the quote, “Essential for some, useful for all,” coined by the W3C’s Web Accessibility Initiative (WAI). It emphasizes andelaborates how the web accessibility features are useful for all users irrespective of disabilities/challenges. For example, all of us would prefer a structured layout to navigate the website instead of searching for information on the webpage. Similarly, having simple, understandable error messages and instructions is a fundamental need for all users. Furthermore, voice-based applications are witnessing rapid adoption across the globe because of the ease they provide in this fast-moving world.

Now, shifting the focus to a business’s point of view, the disabled community forms the third-largest economy globally in purchasing power 1 , as 1 in 5 among the population is challenged in some way. This results in a concrete business case to invest in web accessibility features.

Furthermore, an accessible web is a legal requirement in several countries. According to the United Nations Convention on the Rights of Persons with Disabilities (UN CRPD), access to information and communications technologies, including the web, is defined as a fundamental human right. 2 Every country now has legal policies for web accessibility based on that. We can also see a surge in the lawsuits against Fortune 500 companies, since 2017, for violation of those policies. The first case was won in 2017 when a person with visual impairments sued Winn-Dixie, a supermarket chain in the US because their site did not support screen readers! 3 So, if you aren’t already, it is time for the software development teams and businesses to pay closer attention to web accessibility features.

This chapter will give you the breadth and depth of web accessibility, focusing on accessibility testing strategy and tools. You will get an overview of the accessibility user personas, the ecosystem of tools andtechnologies, the inner workings of screen readers, and the web accessibility guidelines that are mandated by many governments around the globe. You will also learn about the web development frameworks that support accessibility, and a shift-left accessibility testing strategy. Finally, there are exercises on the automated accessibility auditing tools that you can incorporate in your continuous testing strategy and empower your team to continuously deliver an accessible website!

Building Blocks

Let’s begin with getting to know the accessibility user personas and their specific needs. The user persona discussion will be followed by an overview of the accessibility ecosystem and the web accessibility guidelines.

Accessibility User Personas

To recall, a user persona is a character that represents a set of larger audiences with similar attributes. We create user personas in software projects to understand their specific needs and imbibe them throughout the software development stages, starting from design. So, here is a set of accessibility-specific user personas.

Frameworks

  • Matt, a 30-year-old business professional, has broken his arm recently. As he struggles to operate the mouse, he needs keyboard- only access to the website.

  • Helen is an 80-year-old retired teacher whose color sensitivity has become poor. To access the web, she needs color contrast on the UI, i.e., distinguishable background and foreground elements like images, links, buttons, etc. This requirement also fits users with color blindness.

  • Abbie is a teenager who has cognitive disabilities. Since she takes time to learn new things, she needs a clean web layout with proper headings, navigation bars, and consistent navigation structures to access the web. Also, Fred, who wants to check out the nearby gas stations while driving, needs a clear layout of information to make a decision quickly.

  • Connie is blind and an independent store manager. He needs text to speech and voice recognition support for accessing the web. Also, Lakshmi has an infant whom she carries around most of the day. She needs speech to text in order to send any texts.

  • Maya is a software professional who has reduced dexterity, and she needs large texts, buttons, and controls to access the web. Users with dyslexia and low vision also need this requirement.

  • Philip is deaf and a cooking enthusiast. He needs captions to understand the recipe videos.

  • Xiao is a Chinese-speaking retail shop owner. He has been learning English for only a couple of months now. Xiao needs simple instructions, understandable content that doesn’t include jargon, complex words, and sentences to access the web. Users with cognitive and learning challenges also would benefit from this feature.

To consolidate our user personas, they displayed visual (complete or partial), hearing, cognitive, and muscular challenges. They also exhibited temporary challenges. So the goal is to enable all of them to perceive, understand, navigate, and interact equally, like any other user.

Accessibility Ecosystem

To build accessible web features, we have to understand the entire accessibility ecosystem. Ecosystem, here, refers to the various tools and technologies beyond just web technology that interact with each other to deliver content to the challenged. For example, a user persona like Connie, who is blind, uses text to speech and voice commands to interact with the web. To enable that, the text reader technology and the voice command technology cooperate together. Some of our other personas need assistive devices to be integrated to the computer. So, in order for us to think of different accessibility use cases to build, we need to understand these components and integrations, at least on a high level. Here is a brief on the accessibility ecosystem:

Web Development Tools & Practices

Quite obviously, the web development tools such as HTML, CSS, etc., should have the necessary facilities to make the web accessible. For instance, to pass on the information about the elements on the page to the screen reader, there should be provisions in the web development frameworks to mention them explicitly.

User-Agents

These are the tools that render the web content, such as the browsers and media players. These user agents should understand that the web content is enabled with accessibility-related features and should integrate with a screen reader, for example, to hand over the content.

Assistive Technologies

The additional devices and technologies that talk to the browser and relay the information back and forth to the user are the assistive technologies — for example, the screen readers, alternative keyboards, switches, and more.

As we can see, the ecosystem comprises a vast set of tools and technologies. Having accessibility provisions in all these components enables Connie and others to interact with the web. There are possibilities that one of these components is ahead with features, and the others are still catching up, which leads to more round-about work in one area or a lack of some features for our user personas.

However, to ensure all these components have standardized accessibility features, W3C WAI has established international standards for each of them, as listed here:

  • Authoring Tool Accessibility Guidelines (ATAG) establishes the standards for content authoring tools such as HTML editors.

  • Web Content Accessibility Guidelines (WCAG) defines web content standards and is the one we should pay attention to during development.

  • User Agent Accessibility Guidelines (UAAG) address web browsers and media players’ standards, including some aspects of assistive technologies.

All of these standards are detailed on the WAI site. As web development teams, we will deep dive into Web Content Accessibility Guidelines (WCAG) in the next section, specifically version WCAG 2.0. WCAG 2.0 lists down specifications for all the web content (text, image, color, media, etc.) to make them accessible. Many countries have policies crafted for government, public, and private sectors to mandate WCAG 2.0 standards, as mentioned earlier.

Screen Readers

To make sense of why WCAG 2.0 prescribes specific guidelines, we need to briefly understand how an assistive technology such as a screen reader works. To recollect, our visually challenged user personas use screen readers.

First, as the name suggests, screen readers read aloud the content on the page for the user, and they interact back with the website via keyboard. So, as they hear the content, the users press keyboard shortcuts such as Tab, Tab+Shift, Enter, etc., to interact with the site.

The screen readers recite the content on the page in the order of the page’s ‘accessibility tree.’ The accessibility tree is a DOM-like structure on the web page. It has the page elements with attributes like roles, IDs, etc., explicitly defined in a sequence representing a meaningful flow. For example, consider a booking site with ‘To’ and ‘From’ text input fields and a ‘Search’ button on the homepage. The accessibility tree will be structured to represent the ‘search tickets’ user flow, i.e., to enter the ‘From’ location first, then the ‘To,’ and then click on the ‘Search’ button. We can code the elements on the web page to be hidden in the accessibility tree if needed.

It is good to experience screen readers to relate to the accessibility features better. Google Chrome has an extension called ‘ChromeVox Lite,’ which is a browser-based screen reader. Try it out. Additionally, there are demo websites that give a sense of how the visually impaired might experience the web. Figure 8-2 shows the home page of one such example booking site, where it is intentionally blurred and lets you finish the booking using ChromeVox Lite and keyboard access.

Frameworks

WCAG 2.0 - Guiding principles & Levels

By now, if you had a chance to try the screen reader, good! Otherwise, I hope you got an idea of how things work from the previous section. Let us explore WCAG 2.0 in detail now.

WCAG 2.0 sets four guiding principles to remember while designing web content. They are: the content should be 'perceivable,’ ‘operable,’ ‘understandable,’ and ‘robust.’ WCAG 2.0 has carved out three levels under each of these principles called Level A, AA, and AAA.

Level A

The essential support without which the site is inaccessible, such as audio or video content, should have captions, color contrasts, keyboard navigation, etc., that will enable all our user personas to navigate the web.

Level AA

This encompasses Level A requirements plus a few additional stricter requirements, such as constrained color contrast ratios across the site. Some legal policies recommend this level.

Level AAA

This level again subsumes all the requirements from the above two levels and calls for additional enhanced requirements to make the web truly accessible for all users. An example requirement from this level would be to have sign language interpretations for video content. Mostly, when you seek to achieve this conformance level, it shows the users you really care about them!

Organizations must choose the level they need to conform to based on legal requirements.

Level A Conformance Standards

Let’s understand the WCAG 2.0 Level A requirements, the minimum level the websites need to adhere to. The requirements will become our accessibility test cases directly.

Perceivable

The first principle is to make the web content available conveniently for all our user personas. Only when they can perceive the content will they be able to operate on them further. So we should think of all scenarios that can hamper this essential requirement right from the web design phase and avoid them.

  • WCAG 2.0 elaborates this principle with detailed requirements for us to begin with, as follows:

  • All non-text content, like images, should have an alternate text that describes them to enable visually challenged users to understand the content using screen readers.

  • Audio or video content should have text transcripts and captions as alternatives—synchronize captions with the media. Give users the provisions to pause, stop, and control volume additionally.

  • Any audio that starts playing automatically on page load should have audio control mechanisms like pause, replay, and volume controls.

  • Design the web page’s information and structure to have a hierarchy, such as a page title above all headings, appropriate pagetitle and heading tags, and so on. This helps users with screen readers to have a meaningful flow.

  • Instructions to navigate the website should not solely rely on sensory characteristics of components such as shape, color, size, visual location, orientation, or sound. For example, avoid an instruction that says ‘wait till the button turns green’ or ‘wait till you hear a beep.’

  • Colors should not be the only way to indicate an action on the screen. Make it intuitive via text for color-blind users.

  • The page should have color contrast from background and foreground elements to support users with less color sensitivity. There is a fixed ratio prescribed for this.

Understandable

The web may have many elements, and user flows to complete an action. For example, to book a flight ticket, there may be several steps and instructions. Such content and user flows should be carefully crafted to be simple and straightforward for all user personas. WCAG 2.0 once again calls out solid requirements here, as seen here:

  • Avoid jargon and technical terms; present simple, meaningful content instead. For example, avoid technical error messages like “034506451988 is invalid”. Instead, “Incorrect Date format” is understandable.

  • Provide expansions and abbreviations where necessary. Avoid sudden changes in context (e.g., opening multiple windows) as it will affect the keyboard navigation.

  • Avoid change of context when the user has different settings like larger font size.

  • Provide clear, actionable label text for elements to help users take the right action. For example, the email address input field should have the label ‘Email’ and sample as example@xyz.com.

Robust

Finally, we should make the web content robust to support different types of user agents and assistive technologies, i.e., screen readers are not the only assistive technologies, but many others will need proper integration. WCAG 2.0 calls out the below requirements for this principle:

  • The markup language content should follow standards like having opening and closing tags, no duplicates, unique IDs, etc., so that it is easily parsable by multiple assistive technologies.

  • Name, role, and state of elements, including those generated by scripts, should be available for assistive technologies. Example, role=”checkbox” and aria-checked=”true|false”. Provide the updated state of the checkbox to the screen reader after selection.

Accessibility Enabled Development Frameworks

To build the earlier mentioned features, many development frameworks provide elaborate accessibility support. For example, React fully supports building accessible websites, often by using standard HTML techniques. Similarly, the Angular Material library, which the Angular team maintains, is a suite of reusable UI components that aims to be fully accessible. And Vue.js has support to create accessible components as well. Similarly, there are automated accessibility auditing tools, which we shall see in the testing strategy next, that alert if standard accessibility-related tags are missing in the HTML. So, stay relieved that you got support! Your team can achieve this without much additional effort too.

Accessibility Testing Strategy

It should be evident from the above sections that most of the accessibility requirements have to be thought through right from the start of the project and continuously supported throughout the development process rather than retrofitted post the testing phase. For example, to conform to Level A, you should incorporate simple, consistent navigation across the site, captions for videos, meaningful error messages, color contrasted images, etc., during product design rather than during development or testing. So, as a first step to support accessibility, teams should define the application’s accessibility user personas, similar to what we discussed at the beginning of the chapter, and tailor user stories catering to each of those user personas. And, when your team discusses the product’s features, collectively validate if the accessibility flows are included in the scope. That will be the primary step in shifting accessibility testing to the left-most!

Figure 8-3 shows the shift-left implementation of accessibility testing throughout the software development cycle. Let’s delve into them one by one.

Frameworks

Accessibility Checklist in User Stories

We saw a lot of common requirements that span across all the pages of the website as part of the WCAG 2.0 guidelines. For example, adding an alternate text, keyboard navigation, page title, etc., are universal requirements across all pages. Hence, appending an accessibility checklist to all the user stories will help developers and testers go through these requirements meticulously. Here is a generic checklist to which you can append your application-specific list and use it in your teams.

ACCESSIBILITY CHECKLIST

  • Check for the ‘page title’ on the browser. When you hover over the browser tab, you can see the page title as a small widget in Chrome. This text should clearly define the context of the page on the website.

  • Check the basic structure of the web page to have proper element attributes and hierarchy. Turn off the CSS and ensure the order of the elements listed to be screen reader-friendly. Chrome has an ‘accessibility tree’ view in the dev tools to show the order of elements.

  • Check for keyboard-only navigation. Check for proper element highlights and keyboard operations for backward, forward, and exit when doing that.

  • Check for error messages, labels, links, and in general, any text on the page to indicate the right intent.

  • Check the page’s readability on resizing the text, either using system preferences or browser zoom options.

  • Check for readability in grayscale. On Mac, enable this setting from System Preferences -> Accessibility -> Display -> Use grayscale.

  • Check the captions for video and audio content to be meaningful and synchronized.

  • Check for meaningful alternate text descriptions of images. Verify this by turning off the image download option in browser settings. In Chrome, select Settings -> Site Settings -> Images -> Block will turn off image download. Then the browser shows the alt text in place of images.

  • Check screen reader flow is meaningful and is able to complete the action.

Automated Accessibility Auditing Tools

The accessibility checklist includes some of the aspects that it is only possible to verify with human intervention, such as verifying grayscale, zooming in and out, checking the meaning of the alternate text, and so on. You can complement it with automated auditing tools that will scan the basic HTML structure and alert if the respective accessibility tags for elements are missing. These save a lot of time and effort by giving instantaneous feedback on missing tags during the development phase itself!

The tools come as static-code analyzers and run-time accessibility checkers. eslint-plugin-jsx-a11y is a linting tool for React. The ESLint plugin enforces several accessibility standards directly in your JSX. Similarly, Codelyzer has linting rules for accessibility standards in TypeScript, HTML, CSS, and Angular source code. These tools will give feedback as you’re developing. And the run-time checkers such as axe-core, pa11y-ci, lighthouse CI, which are discussed later in the chapter, will give feedback on the actual web page post-development. They can be run in the local development machine to get faster feedback on the pages developed as part of every user story and also as part of CI for continuous testing.

Apart from these, functional flows that cater to accessibility needs, such as having a separate transcript section below the video/audio or a set of meaningful error text and instructions, can be added as automated micro and macro level functional tests.

Manual Testing

Manual testing is going to be critical in validating website accessibility. As mentioned before, the automated tools only check for the HTML structure, and the checklist only has the mandatory items that are spread across pages. There will be many items left to handle apart from these as part of manual testing — for example, verifying the functional flow with a screen-readerand a keyboard. So as part of manual testing, you can focus on different scope in different stages such as the following:

User Story Testing

As part of user story testing, ensure the checklist works appropriately on all the pages where the user story expands. You can couple it with a web accessibility evaluation tool, WAVE, a free online service by WebAIM. It helps find accessibility issues on the web page as per WCAG 2.0 guidelines. Although the checks do not verify all the WCAG 2.0 standards’ requirements, it highlights issues visually on the web page on a browser. This helps in noticing some of the accessibility issues that we may not be aware of. Alternatively, you can also choose Google LightHouse as part of Chrome DevTools to get accessibility audits on the web page using the Chrome browser. Both of these tools are discussed later in this chapter.

Feature Testing

With story-level testing, you would cover the bulk of the accessibility testing effort. When a feature is complete, you need to do another round of manual testing to ensure end-users can complete the user flow only with keyboard access and if the screen readers can navigate the functionality as expected.

This level of feature testing will help identify the lack of coherence in the application’s end-to-end navigation. To test keyboard-only navigation, use the ‘Tab,’ ‘Tab+ Shift’ keys to move forward and backward through the website, ‘Enter’ key to select, and ‘Up’ and ‘Down’ arrow keys for drop-down selection. While doing this, make sure the focus is on the right element with good highlights.

For testing the screen reader flow, you can use Mac OS’s inbuilt screen reader called the VoiceOver, which can be turned on from System Preferences -> Accessibility -> VoiceOver-> Enable. VoiceOver has tutorials to help you learn screen reader functionalities. So, with these checks, you will finish the end-to-end accessibility testing of features.

Release Testing

Finally, when all the release features are completed, it is recommended to do testing with actual end-users, including people with challenges. Since different users may have various assistive devices, this will give you real-time feedback before the product goes for a final evaluation for conformance certification. UserTesting.com is one such services company that you can request for people with challenges for user testing.

Conformance Certification

Once the website is ready, experts in WCAG standards do the conformance evaluation before going live. It is not a centralized unit. Organizations may even have in-house experts or hire from the market to do the final assessment once the product is ready for certification.

Since every single element on the page requires changes to make them accessible, shifting-left is the only way to rescue your team from the uphill task of fixing all the accessibility issues at the end of the development cycle.