Accessibility Inspections Using the Web Content Accessibility Guidelines by Novice Evaluators: an Experience Report

The use of Web Accessibility standards is fundamental as a first step in evaluating the accessibility of digital resources. The Web Content Accessibility Guidelines (WCAG) have become the leading worldwide reference for accessibility standards and are used by many governmental regulations in numerous countries. However, many studies have revealed difficulties novice evaluators encounter in conducting accessibility inspections using WCAG. In Brazil, there is ongoing work to adopt WCAG 2.2 as the basis of a national standard by the Brazilian Technical Standards Association (Associação Brasileira de Normas Técnicas - ABNT), which will include a checklist to simplify the use of the guidelines. This experience report describes the experiences of two groups of students engaged in an outreach digital accessibility project, aimed at providing detailed inspection reports to entities representing people with disabilities when filing official complaints concerning digital accessibility. The first group of students evaluated websites using the WCAG 2.2 documentation alone. The second group had their first experience conducting accessibility inspections using the checklist and guidance from the ABNT checklist under construction. The report provides insights into the main difficulties encountered by the students in both scenarios and the strategies they used to overcome them. The experience report provides a detailed account of the types of success criteria that presented the most difficulties and how the new checklist provided better guidance to them. The reflections in the report provide lessons learned concerning using the ABNT checklist and possibilities for improvements for both the checklist and future versions of WCAG.