What is Accessibility testing?
Accessibility testing is the special branch of testing dedicated to making sure that the newly developed Web application/Mobile App/ PDF are accessible to everyone including people with disabilities. We may also say that it is a subset of “Usability Testing”.
Why Accessibility testing is necessary?
According to world health organization, “Over 1 billion people live with some form of disability and almost everyone is likely to experience some form of disability- temporary or permanent ─ at some point in life.” Hence, it is essential to make sure our application is accessible to all the users irrespective of whether they have any disabilities or not. Many laws require that any software application be accessible by disabled people.
Laws and compliances related to accessibility
- Web Content Accessibility Guidelines (WCAG): The World Wide Web Consortium (W3C) created the Web content accessibility guidelines depending on 4 principles.
- Information should be perceivable
- Information on a website should be operable
- All information should be understandable
- A website should be Robust
- The WCAG 2.0, 2.1 guidelines have
- 12 general guidelines
- 61 testable criteria
- 3 level of conformance (A, AA, AAA)
- You can find more information on WCAG at this link: https://www.w3.org/WAI/standards-guidelines/wcag/
- American with Disability Act (ADA): ADA is a civil rights law that states that all public entities public buildings, transportation, and organizations should make the technology accessible to everyone.
- Rehabilitation Act, section 504 and section 508: Section 508 covers access to and use of electronic information technology. Section 504 requires agencies to provide individuals with disability an equal opportunity to access the workplace, education & other organization
Types of Disabilities to consider
Below are various kind of disabilities that must be considered while developing and testing for Accessibility
- Visual Impairments: This can include complete blindness, various common types of low vision and poor eyesight, various types of color blindness.
- Seizures: Photo-epileptic seizures caused by visual strobe or flashing effects
- Motor/Mobility: This includes a difficulty or inability to use hands, legs, muscle slowness, loss of fine muscle control due to conditions such as Parkinson’s Disease, muscular dystrophy, cerebral palsy, stroke.
- Auditory: Deafness or hearing impairments, including individuals who are hard of hearing.
- Cognitive/Intellectual: Developmental disabilities, cognitive disabilities of various origins learning disabilities (e.g., dyslexia), Cognitive disabilities affecting attention, memory, problem-solving and logic skills, etc.
Accessibility Testing Tools
Testers need to use both Automated and Manual tools to make sure the application is accessible or as we may say, WCAG compliant. However, it does not matter what tool is used to check for compliance if the application meets the WCAG guidelines.
Automated Testing Tools
Accessibility testing is a time-consuming and lengthy process, so it is natural to get tempted to use a few automated tools to test the accessibility. There are a few excellent open-source tools like Wave and Axe in the market which can be used to catch accessibility issues, but automated tools can detect only 30% – 50% accessibility issues. They may also generate false positives or raise warnings that need interpretation. Hence, they are insufficient for confirming accessibility. Still, these are valuable tools in a developer’s workflow and finding the programmatic error.
One or more of these automated tools/ browser extensions can be used to perform the automated accessibility testing.
Browser Extensions:
- WAVE – Chrome or Firefox extensions (link)
- Axe – Firefox extension (link)
- Axe – Chrome extension (link)
- Siteimprove – Chrome or Firefox extensions (link)
- Lighthouse – Chrome extension (link)
- Microsoft’s Accessibility Insights – Chrome or Edge extensions(link)
Manual accessibility testing tools
Although usage of the above tools is helpful in accessibility testing, Manual testing is also essential to simulate real user (with disability) experience. The below tools can be used to simulate and test real user experience
Screen Readers
The visually impaired users use one of the below screen readers to navigate through the applications, so it is important for the tester to learn the basic operations of screen readers for testing. Major screen readers include
- VoiceOver (default on Mac) – VoiceOver Getting Started Guide (apple.com),
- NVDA (free to download on Windows) – Link to download NVDA – https://www.nvaccess.org/download/
- JAWS (available for purchase on Windows) – https://support.freedomscientific.com/Downloads/JAWS
- Narrator (default on Windows accessibility) – https://support.microsoft.com/en-us/windows/complete-guide-to-narrator-e4397a0d-ef4f-b386-d8ae-c172f109bdb1
JAWS is used by most of the Visual Impairment users as the most popular screen reader. However, NVDA is an open-source tool and is gaining a lot of popularity recently.
Color Contrast Analyzer
Although most of the automated tools validate the color contrast issues, they investigate CSS on a page. It does not check for the text on top of a background image. So, it is important to use the below google chrome extension.
Steps to follow while doing the Accessibility testing
Step 1: Test with an automated tool
Install and run the Wave and/or axe tools (links to install the browser extension are available in Accessibility Testing Tools section) for all the pages of the application. Wave and Axe tool will normally be able to find 30-50% of the accessibility issues.
It is advisable that developers should also install these tools and cover it as part of unit testing as these tools also help in fixing the issues.
Step 2: Keyboard only testing
The second step in Web accessibility testing is to disable the mouse and use only keyboard to navigate through the application including all Menus, pages, and interactive elements. People with mobility/motor disabilities find it hard to use the mouse, thus they navigate through the pages of the application using only Keyboard controls. Here are the major scenarios that should be tested using keyboards
| Test scenarios | Test case/ Use case |
| Key Board Focus/ tab order/ Electronic forms | Users should be able to TAB through all the pages of the application to see if the page order is logical. |
| Users should be able to navigate through the page using only TAB, ENTER, SPACE, and UP and DOWN arrow keys from the keyboard. | |
| Users should be able to reach all the elements on the page including links, dropdown menu items, buttons, and other interactive elements. | |
| The user should be able to navigate and select an item from the dropdown menu. | |
| If the users take a longer time to fill in the form or if the page is refreshed (auto refreshed), the data entered is retained. | |
| When the user left a required field blank, keyboard focus shifts to an error message or the field | |
| Or if the user enters improperly formatted data to a certain field (for example alphabets on a numeric field), keyboard focus shifts to the error messages | |
| Pop Up messages | When a pop-up message is displayed, the keyboard focus moves to the pop-up menus |
| The user should be able to cancel/ close the dialog messages using the keyboard. | |
| When the user closes the dialog boxes, Focus returns to a logical location. | |
| Multimedia – features such as videos, audio files, calendars, Flash content, and photo carousels. | User can access the available multimedia files on the application using the keyboard |
| Controls can be activated using the keyboard; Controls can be tabbed through |
The above scenarios are derived from the below WCAG 2.0 guidelines
2.1.1 – Keyboard, 2.1.2 – No Keyboard Trap, 2.2.1 – Timing Adjustable, 2.2.2 – Pause, Stop, Hide, 2.4.7 – Focus Visible, 3.3.1 – Error Identification, 3.3.2 – Labels or Instructions
Step 3: Testing Non-Visual Navigation with a screen reader (NVDA/ windows Narrator/JAWS)
People with Visual Impairments use screen readers to navigate through the applications. While using a screen reader, users navigate a page is by using the keyboard (TAB, ENTER, Up/Down Arrow keys). So, it is important to make sure our application can be navigated in a logical order by the screen reader and all the elements are identified appropriately by the screen reader. Here are the major scenarios that should be tested using the screen reader
| Test scenarios | Test case/ Use case |
| Descriptive text | User/ screen reader should be able to differentiate between different links – Links should be descriptive, without generic text such as “click here”. |
| User/ screen reader should be able to differentiate between different Error messages, Error Messages should be descriptive and clear | |
| All images (except the decorative images) should have a descriptive alternate text and is identified by the screen reader | |
| User/ screen reader should be able to identify each Button uniquely. Each button on the page should have a descriptive alternate text and should be identified by the screen reader | |
| Logical Heading | The web page should be structured by Headings. All Headings should be logical and should describe the importance of the content. |
| The screen reader user can navigate through all the headers – Make sure none of the headings are skipped by the screen reader. | |
| Screen reader user can interpret data tables properly – Data tables should have a proper header for column rows. | |
| The screen reader can each frame by its descriptive titles. | |
| Forms | While filling out a form, the user/screen reader can identify each field by its descriptive labels. Each field o the form has descriptive labels |
| User should be able to fill out the forms and submit them successfully with the screen reader | |
| When the user missed to fill out a required field, the screen reader is directed to the error text and dialog buttons. | |
| When an error dialog is dismissed or closed, the control of the screen reader returns to the empty field automatically. | |
| All the buttons on the form have a descriptive label and read correctly by the screen reader. | |
| Captcha | The user should be able to access CAPTCHA by the keyboard. |
| The screen reader can access and read CAPTCHA. | |
| A screen reader can fully access the Audio CAPTCHA, including a pause that allows the screen reader to finish before the audio begins | |
| Audio CAPTCHA should have an alternative for users with hearing impairments | |
| Multimedia | All multimedia controls on the application have the alternate text and are read properly by the screen reader. |
| User can control when to play the Audio/ Video, and it is not played automatically |
Below are the WCAG 2.0, 2.1 guidelines which are referred to derive the above scenarios
1.1.1 – Non-text Content, 1.2.1 – Audio-only and Video-only (Pre-recorded), 1.2.2 – Captions (Pre-recorded), 1.2.3 – Audio Description or Media Alternative (Pre-recorded), 1.2.4 – Captions (Live), 1.2.5 – Audio Description (Pre-recorded) , 1.3.1 – Info and Relationships, 2.1.1 – Keyboard, 2.4.1 – Bypass Blocks, 2.4.4 – Link Purpose (In Context), 2.4.6 – Headings and Labels, 3.3.1 – Error Identification, 3.3.2 – Labels or Instructions
Step 4: Testing Alternate Visual Access
Users with Color Blindness cannot distinguish between certain colors. And users with low vision will not be able to clearly see certain color contrasts. Also, users with low vision change zoom settings to 200 – 300% to be able to read the content.
Color contrast issues can be checked by WAVE tool or the chrome Color Contrast Analyzer extension. Apart from that the below scenarios should be manually verified using Zoom feature or magnifier tool (windows ease of access tool)
| Test scenarios | Test case/ Use case |
| Choice of color | A color-blind user or screen reader will not be able to interpret the information if color used as the sole means of conveying information on the page. Verify that color is not used as the sole means of conveying information on the page. |
| Font size/ Zoom | Low vision users should be able to increase the font size on the page by using the Zoom feature. |
| Verify that the text does not become pixelated when zooming to 200% – 300% in on the page. | |
| Other Items on the page such as pictures, tables do not become jumbled when zoomed. |
Below are the WCAG 2.0, 2.1 guidelines which are referred to derive the above scenarios
1.4.1 – Use of Color, 1.4.3 – Contrast (Minimum), 1.4.4 – Resize Text, 1.4.5 – Images of Text
Step 5: Usability
Many users have Cognitive/Intellectual disabilities, and they might find it difficult to process complex information. Major scenarios that should be covered to ensure that the web page has clear content and logical navigation are –
- Fonts on the webpage are basic, clear, and easy to read.
- The information on the page/ certain sections is not crowded or chaotic.
- There is no flashing, blinking, or moving content on the page.
- Menus and navigations are consistent and logical across the entire application
The below WCAG guideline are referred to derive the above scenarios–
2.2.2 – Pause, Stop, Hide, 3.2.3 – Consistent Navigation, 3.2.4 – Consistent Identification
Step 6: Testing Non-Audio Access – Testing for hearing impairments
For users with hearing impairment, we need to make sure that all the Audio and Video have caption/ transcripts which can be read clearly to understand the content of the Video/Audio. Major scenarios to be tested for this category are –
- Users have the ability to turn on the caption for Videos
- All existing captions are accurate and match with the content of the video
- Captions are clearly visible.
- Users should be able to access the transcript for the audio files.
Applicable WCAG 2.0, 2.1 references for this category
1.2.1- Audio-only and video-only (Pre-recorded), 1.2.2 – Captions (Pre-recorded), 1.2.3 – Audio Description or Media Alternative (Pre-recorded), 1.2.4 – Captions (Live), 1.2.5 – Audio Description (Pre-recorded)
Step 7: Downloadable Files
Any documents which are downloadable from the application/ webpage should also be accessible. Here are the major scenarios that must be considered while testing the downloadable files
- The downloadable files (PDF, PPT, Word Document) have a logical reading order
- The downloadable files (PDF, PPT, Word Document) have alt text on non-text elements.
WCAG/ ADA testing on PDF
You can find more details of PDF Techniques for WCAG 2.0 at the below link
An easier way to test PDFs for accessibility is to use the in-built accessibility test
Step 1: Install Adobe Pro DC on your computer and log in using the below credentials.
Step 2: Open the required PDF.
Step 3: Search for Accessibility in the right search tool
Step 4: Run accessibility check
WCAG/ ADA testing on Microsoft Word/PPT/Excel:
Like Adobe Pro DC, all Microsoft products have accessibility tests inbuilt under Review > Check accessibility. Below screenshots show how to access the accessibility features on a Microsoft word document.
Conclusion
Accessibility is a vast topic; this article covers the basics that a beginner in accessibility testing would require to get started. Accessibility testing is very interesting when we understand why we need to make our Webpage/ Applications accessible.
