As a web accessibility resource hub, it was only fitting that Access iQ™ factored accessibility into the design and development of our website from the planning stages and engaged an accessibility testing specialist in the design and testing phases.
Automated evaluation tools can only check about 30 per cent of the elements that require testing so there is no replacement for human testing — and testing with users who have disabilities will give you the most accurate results, thus providing you with the best user experience.
Despite having the accessibility know-how, we chose to outsource user testing to a specialist in the area since they have the testing tools and access to testers with disabilities that ensure the most accurate sort of testing environment.
Developed in Drupal 7, the Access iQ™ website aims to comply with WCAG 2.0 Level AA, though meeting these requirements doesn't guarantee your site won't still incur barriers to access.
Overall, deciding on this approach and engaging an external specialist saved us time and money, and both organisations learned a lot in the process.
How the testing was conducted
Accessibility testing specialists Web Key IT were engaged to conduct the website accessibility audit of the Access iQ™ website releases. While Web Key IT were also consulted during the design phases to test aspects of the design against WCAG 2.0, the audit conducted during development consisted of two types of testing:
- A technical audit of the different phases against WCAG 2.0 Level A — AAA criteria.
- Testing by a team of trained user testers, all of who have disabilities.
Note: While the technical audit included testing of WCAG 2.0 Level AAA requirements, the intended level of compliance for the Access iQ™ website is to meet Level AA.
Web Key IT has a unique strategic partnership with the Digital Accessibility Centre (DAC) in South Wales, UK. The DAC is a not-for-profit social enterprise with a team of ten trained and experienced user testers with disabilities. Their disabilities include but are not limited to:
- Low vision
- Colour blindness
- Mobility limitation
- Asperger's Syndrome
The team at DAC has been conducting accessibility audits for two years, though most of the team has previously worked together and have been auditing for ten years. They have tested some of the largest websites in the UK, including Channel 4 who held the broadcast rights to the 2012 Paralympics Games.
Accessibility user testing
The W3C WAI has been working on the development of the Evaluating Web Accessibility resource suite. The first part of this suite provides a number of resources that are currently available or under development. One of these, the Web Accessibility Conformance Evaluation Methodology (WCAG-EM) was released on 20 September 2012 and open for public comment until 20 October 2012. This Methodology provides specific mention to the need to involve users in website accessibility testing.
The second part of the resource suite deals with this aspect of involving users in testing websites. In particular, this resource describes the various activities that would benefit from the involvement of users with disabilities including older users.
Testing with people who have disabilities
Users with disabilities are able to assist in all areas of a website's development and testing from the preliminary planning when they are able to tell the developer the most common problems they encounter on the web through to final product testing or as part of the audit of an established website or application.
This testing group will usually be asked to perform tasks that would be common to users of a website — referred to as a script. For instance if this website were for a local government, the users might be asked to see if they could pay their shire rates, get information on fencing policies, licence a dog, find the date for the next council meeting and find the location for their nearest public library.
The users then attempt these tasks with adaptive strategies such as the ability to increase text size, and their own particular assistive technology, such as screen reader, Braille reader, voice-activation software, text enlargement software, etc. Experienced user testers may also be asked to test certain WCAG criteria such as colour contrast, heading structure etc.
Research by DAC shows that it is best for accessibility testing to be conducted in teams: a user with assistive technology who attempts to complete the scripts and a support person to make note of any issues.
For instance a screen reader user may not be able to locate a video due to lack of controls on the video, or they might not be able to complete a form due to lack of keyboard handlers. If the tester conducts this test remotely or independently, these omissions would not be noted.
The support person working with the testers is able to make note of the various issues encountered by the tester as well as providing assisting or direction where required. The testers take screen shots of issues they encounter and describe the problems in their own words.
This information is provided to the website owner by way of a report from the user tester.
Testing phase one of the website template
Objective: To test whether the initial phase of the website is usable by people with disabilities.
This testing will pave the way to understanding the needs for all of the following phases as it includes the templates for the website. The users were given free rein to see what problems they could find that it would pose barriers for their use of the website with their particular assistive technology. The testers were asked to describe the problems and make recommendations where appropriate.
In the following profiles, you will meet the four accessibility testers who describe their interaction with the first phase of the Access iQ™ website templates.
Rebecca Morgan: Voice activation analyst
Rebecca Morgan is a voice activation analyst who tests websites with Dragon Naturally Speaking speech recognition software. She has a BA (Hons) in Education Studies and Social Inclusion, and has completed a Preparation to Teach in the Lifelong Learning Sector (PTLLS) course in online teaching.
Testing: Links to ensure all hyperlinks have been tagged and aligned properly.
View Website accessibility testing — links (close captioned) on YouTube.
- The first two tags are for invisible links — there are two skip links that only appear when tab-activated.
- Links to social media are too close together so the user can't differentiate between them.
- The carousel changes with no pause button which means the voice activation user would have trouble getting to the link in the carousel before it changes to the next slide.
- On the landing page of the Create site section, Dragon Naturally Speaking tags some links that the user can't see which causes confusion. This results from drop-down menus; when you open the menu, the links are properly aligned. It is recommended that the expanding links have arrows to show that they drop down, and that the links are tagged so that the user can properly associate them with the controls.
Jaime Purvis: Screen reader analyst
Jaime Purvis is a screen reader analyst. Legally blind, he has been using JAWS for eight years and testing websites with the screen reader for the last six. He feels the job is important because the websites that are tested will eventually be made more accessible for screen reader users, making it easier for blind users to navigate them.
Testing: Form fields to see if screen reader users can access all form fields.
View Website accessibility testing — form fields (close captioned) on YouTube.
- There is text in the edit field that doesn't disappear on focus. If the user puts in a number, say 26, they end up with 126 meaning that the '1' in the form doesn't disappear automatically. It is recommended that the text is made to disappear when the user enters the 'forms mode'.
- There are two headings at
<h1>level, which causes confusion for the screen reader user when trying to determine the structure of content.
- There were some problems with the naming of links that don't always make sense out of context. They make sense in specific context, but not always when asking for a list of links using a screen reader.
Gary Thomas: Low vision analyst
Gary Thomas is a low vision analyst and has worked in the accessibility field for five years. Gary has diabetic retinopathy that has affected his sight to an extent that he is blind in one eye and has very limited vision in the other. Gary currently uses ZoomText screen magnification software to help him use a computer and navigate websites.
Testing: Low vision and colour contrast.
- Some areas do not meet the WCAG 2.0 Level AA requirements of 4.5:1 for colour contrast e.g. the pink background.
- Some words are truncated when resized, including 'Search' and the 'All sections' box is cut off due to a fixed dimensions.
- When the carousel changes, the text overlayed on the carousel image is cut off when zoomed in.
- On hover/mouseover, the colour change is too difficult for the low vision user to see.
Joy Robey: Dyslexia and colour blindness specialist
Joy Robey has rheumatoid arthritis, fibromyalgia and a cardiac condition. Joy also has dyslexia, colour blindness, and impaired vision. She has passed National Vocational Qualifications (NVQs) level one and two, and has been testing websites for four years, specialising in the areas of dyslexia and colour blindness.
Testing: Readability, colour contrast and dyslexia-related impairments.
- Some areas do not have high enough colour contrast e.g. the pink background.
- Some words are truncated when text is resized to the largest setting, including 'Search' and the 'All sections' box is cut off due to a fixed dimensions.
Lessons learned and actions taken
The user testers found the following problems:
- Some colour contrast issues require adjustment to meet WCAG 2.0 Level AA.
- Some of the links would benefit from renaming so that their purpose is obvious.
- Resizing issues need to be assessed due to text truncation and use of absolute sizing for the boxes.
- The carousel poses problems for some users because they can't access the controls to stop it easily.
- Most pages have two headings at
<h1>level, which confuses screen-reader users.
It is very common for the technical evaluation to identify more errors than those raised by the user testers, as it is not possible to cover every disability barrier and type of assistive technology possible. The technical evaluation assesses each page of the website against the WCAG 2.0 criteria and provides a pass/fail grading which is required to assess conformance with WCAG 2.0. The user testers are aware of the WCAG 2.0 criteria, however they concentrate on looking for access barriers for users with their particular assistive technology.
Even with the above in mind, results of the phase one audit demonstrate a strong correlation between the issues raised by the user testers and the technical evaluation. These issues are currently undergoing remediation, and many of them have already been corrected. Further testing will be conducted when all of the issues raised have been remediated to ensure that other barriers have not arisen.
Want to learn more about WCAG 2.0 and web accessibility?
The Professional Certificate in Web Accessibility, a university-accredited online qualification jointly conducted by W3C member Media Access Australia and the University of South Australia, is a fully assessed six-week program that covers both accessibility principles and techniques. The course provides students with all the essentials needed to achieve compliance with international best practice in accessibility. Accessible documents, among many other aspects of WCAG are covered in Access iQ’s complete guides to web accessibility for content authors, web developers and web designers.