research-article

Public Access

Verifying that web pages have accessible layout

Authors:

Pavel Panchekha,

Adam T. Geller,

Michael D. Ernst,

Zachary Tatlock,

Shoaib KamilAuthors Info & Claims

PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation

Pages 1 - 14

https://doi.org/10.1145/3192366.3192407

Published: 11 June 2018 Publication History

Abstract

Usability and accessibility guidelines aim to make graphical user interfaces accessible to all users, by, say, requiring that text is sufficiently large, interactive controls are visible, and heading size corresponds to importance. These guidelines must hold on the infinitely many possible renderings of a web page generated by differing screen sizes, fonts, and other user preferences. Today, these guidelines are tested by manual inspection of a few renderings, because 1) the guidelines are not expressed in a formal language, 2) the semantics of browser rendering are not well understood, and 3) no tools exist to check all possible renderings of a web page. VizAssert solves these problems. First, it introduces visual logic to precisely specify accessibility properties. Second, it formalizes a large fragment of the browser rendering algorithm using novel finitization reductions. Third, it provides a sound, automated tool for verifying assertions in visual logic.

We encoded 14 assertions drawn from best-practice accessibility and mobile-usability guidelines in visual logic. VizAssert checked them on on 62 professionally designed web pages. It found 64 distinct errors in the web pages, while reporting only 13 false positive warnings.

Supplementary Material

Auxiliary Archive (pldi18main-p202-p-aux.zip)

Appendices, including details on the interpretation of visual logic, full descriptions of each tested assertion and their formalization, and reproductions of the CSS standard's descriptions of line height, margin collapsing, and floating layout.

Download
602.01 KB

WEBM File (p1-panchekha.webm)

Download
111.16 MB

References

[1]

Apple. 2017. Human Interface Guidelines. https://developer.apple. com/ios/human-interface-guidelines/

[2]

Apple. 2017. UI Design Do’s and Don’ts. https://developer.apple.com/ design/tips/

[3]

Apple Developer. 2017. UIKit Framework. https://developer.apple. com/documentation/uikit

[4]

Greg J. Badros, Alan Borning, Kim Marriott, and Peter J. Stuckey. 1999. Constraint Cascading Style Sheets for the Web. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology (UIST’15). ACM, 73–82.

Digital Library

[5]

Jeffrey P. Bigham. 2014. Making the Web Easier to See with Opportunistic Accessibility Improvement. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST ’14). ACM, 117–122.

Digital Library

[6]

Alan Borning, Richard Lin, and Kim Marriott. 1997. Constraints for the Web. In Proceedings of the Fifth ACM International Conference on Multimedia (MULTIMEDIA ’97). ACM, 173–182.

Digital Library

[7]

Matthew Butterick. 2010. Practical Typography. Matthew Butterick Typography, online only.

[8]

Lyndon Cerejo. 2011. A User-Centered Approach to Web Design for Mobile Devices. https://www.smashingmagazine.com/2011/05/ a-user-centered-approach-to-web-design-for-mobile-devices

[9]

Tsung-Hsiang Chang, Tom Yeh, and Rob Miller. 2011. Associating the Visual Representation of User Interfaces with Their Internal Structures and Metadata. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 245–256.

Digital Library

[10]

Tsung-Hsiang Chang, Tom Yeh, and Robert C. Miller. 2010. GUI Testing Using Computer Vision. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). ACM, 1535–1544.

Digital Library

[11]

S. R. Choudhary, M. R. Prasad, and A. Orso. 2012. CrossCheck: Combining Crawling and Differencing to Better Detect Cross-browser Incompatibilities in Web Applications. In 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation. 171–180.

Digital Library

[12]

Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. 2016. Programmatic and Direct Manipulation, Together at Last. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). ACM, 341–354.

Digital Library

[13]

Burns David. 2012. Selenium 2 Testing Tools: Beginner’s Guide. Packt Publishing, Birmingham, UK.

Digital Library

[14]

Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’08/ETAPS’08). Springer-Verlag, 337–340. http://dl.acm.org/citation.cfm?id=1792734.1792766

Digital Library

[15]

Diet. 2000. Basic Act on the Formation of an Advanced Information and Telecommunications Network Society. http://japan.kantei.go.jp/ it/it_basiclaw/it_basiclaw.html

[16]

European Commission. 2016. Directive (EU) 2016/2102 of the European Parliament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv: OJ.L_.2016.327.01.0001.01.ENG&toc=OJ:L:2016:327:TOC

[17]

Barry Feigenbaum and Michael Squillace. 2006. Accessibility Validation with RAVEN. In Proceedings of the 2006 International Workshop on Software Quality (WoSQ ’06). ACM, 27–32.

Digital Library

[18]

LLC Formstack. 2014. Free Section 508 Compliance Checker. http: //www.508checker.com/check

[19]

Amy Fowler. 2017. A Swing Architecture Overview. http://www. oracle.com/technetwork/java/architecture-142923.html

[20]

Free Website Templates. 2017. Free Website Templates. https:// freewebsitetemplates.com

[21]

Sylvain Hallé, Nicolas Bergeron, Francis Guerin, and Gabriel Le Breton. 2015. Testing Web Applications Through Layout Constraints. In Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference on. IEEE, IEEE, 1–8.

[22]

Osamu Hashimoto and Brad A. Myers. 1992. Graphical Styles for Building Interfaces by Demonstration. In Proceedings of the 5th Annual ACM Symposium on User Interface Software and Technology (UIST ’92). ACM, 117–124.

Digital Library

[23]

Melody Ivory and Aline Chevalier. 2002. A Study of Automated Web Site Evaluation Tools. Technical Report. University of Washington, Department of Computer Science.

[24]

Ranjitha Kumar, Jerry O. Talton, Salman Ahmad, and Scott R. Klemmer. 2011. Bricolage: Example-based Retargeting for Web Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’11). ACM, 2197–2206.

Digital Library

[25]

Maurizio Leotta, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2015. Automated Generation of Visual Web Tests from DOM-based Web Tests. In Proceedings of the 30th Annual ACM Symposium on Applied Computing (SAC ’15). ACM, 775–782.

Digital Library

[26]

Hsiang-Sheng Liang, Kuan-Hung Kuo, Po-Wei Lee, Yu-Chien Chan, Yu-Chin Lin, and Mike Y. Chen. 2013. SeeSS: Seeing What I Broke – Visualizing Change Impact of Cascading Style Sheets (CSS). In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST ’13). ACM, 353–356.

Digital Library

[27]

Sonal Mahajan, Abdulmajeed Alameer, Phil McMinn, and William G. J. Halfond. 2017. Automated Repair of Layout Cross Browser Issues Using Search-based Techniques. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2017). ACM, 249–260.

Digital Library

[28]

Jennifer Mankoff, Holly Fait, and Tu Tran. 2005. Is Your Web Page Accessible?: A Comparative Study of Methods for Assessing Web Page Accessibility for the Blind. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05). ACM, 41–50.

Digital Library

[29]

Atif Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing. In Proceedings of the 10th Working Conference on Reverse Engineering (WCRE ’03). IEEE Computer Society, 260–. http://dl.acm.org/citation. cfm?id=950792.951350

Digital Library

[30]

A. Mesbah and M. R. Prasad. 2011. Automated cross-browser compatibility testing. In 2011 33rd International Conference on Software Engineering (ICSE). 561–570.

Digital Library

[31]

Leo A. Meyerovich and Rastislav Bodik. 2010. Fast and Parallel Webpage Layout. In Proceedings of the 19th International Conference on World Wide Web (WWW ’10). ACM, 711–720.

Digital Library

[32]

Mozilla Developer Network. 2017. float. https://developer.mozilla.org/ en-US/docs/Web/CSS/float

[33]

Mozilla Developer Network. 2017. Mobile accessibility checklist. https://developer.mozilla.org/en-US/docs/Web/Accessibility/ Mobile_accessibility_checklist

[34]

National Federation for the Blind. 2016. Blindness Statistics. https: //nfb.org/blindness-statistics

[35]

Tuan A. Nguyen and Christoph Csallner. 2015. Reverse engineering mobile application user interfaces with REMAUI. In Proc. 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) (ASE’15). IEEE, 248–259.

[36]

Thomas Ostrand, Aaron Anodide, Herbert Foster, and Tarak Goradia. 1998. A Visual Test Development Environment for GUI Systems. In Proceedings of the 1998 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’98). ACM, 82–92.

Digital Library

[37]

Jason Pamental. 2014. A More Modern Scale for Web Typography.

[38]

Parasoft. 2017. Web UI Testing. https://www.parasoft.com/capability/ web-ui-testing/

[39]

Pearson. 2017. Making E-Learning Accessible. http://wps.pearsoned. com/accessibility/

[40]

Android Open Source Project. 2017. UI Overview. https://developer. android.com/guide/topics/ui/overview.html

[41]

Murray Rowan, Peter Gregor, David Sloan, and Paul Booth. 2000. Evaluating Web Resources for Disability Access. In Proceedings of the Fourth International ACM Conference on Assistive Technologies (Assets ’00). ACM, 80–84.

Digital Library

[42]

Shauvik Roy Choudhary, Husayn Versee, and Alessandro Orso. 2010. WEBDIFF: Automated Identification of Cross-browser Issues in Web Applications. In Proceedings of the 2010 IEEE International Conference on Software Maintenance (ICSM ’10). IEEE Computer Society, 1–10.

Digital Library

[43]

Manolis Savva, Nicholas Kong, Arti Chhajta, Li Fei-Fei, Maneesh Agrawala, and Jeffrey Heer. 2011. ReVision: Automated Classification, Analysis and Redesign of Chart Images. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 393–402.

Digital Library

[44]

Terry Sullivan and Rebecca Matson. 2000. Barriers to Use: Usability and Content Accessibility on the Web’s Most Popular Sites. In Proceedings on the 2000 Conference on Universal Usability (CUU ’00). ACM, 139–144.

Digital Library

[45]

Ivan E. Sutherland. 1964. Sketch Pad a Man-machine Graphical Communication System. In Proceedings of the SHARE Design Automation Workshop (DAC ’64). ACM, 6.329–6.346.

Digital Library

[46]

Anthony T. 2012. Finger-friendly design: ideal mobile touchscreen target sizes. https://www.smashingmagazine.com/2012/02/ finger-friendly-design-ideal-mobile-touchscreen-target-sizes/

[47]

US DOJ. 2010. Department of Justice Advanced Notice of Proposed Rulemaking, RIN 1190-AA61. https://www.ada.gov/anprm2010/web% 20anprm_2010.htm

[48]

US DOJ. 2017. ADA Best Practices Tool Kit for State and Local Governments. https://www.ada.gov/pcatoolkit/chap5toolkit.htm

[49]

Christopher J. van Wyk. 1982. A High-Level Language for Specifying Pictures. ACM Trans. Graph. 1, 2 (April 1982), 163–182.

Digital Library

[50]

Minh N. Vu and Susan Ryan. {n. d.}. 2017 Website Accessibility Lawsuit Recap: A Tough Year for Businesses. https://www.adatitleiii.com/2018/01/ 2017-website-accessibility-lawsuit-recap-a-tough-year-for-businesses/

[51]

W3C. 2008. Web Content Accessibility Guidelines 2.0. https://www. w3.org/TR/WCAG/

[52]

W3C. 2011. Cascading Style Sheets Level 2 Revision 1 (CSS 2.1) Specification. https://www.w3.org/TR/2011/REC-CSS2-20110607/

[53]

W3C. 2016. WAI-ARIA Overview. https://www.w3.org/WAI/intro/aria

[54]

W3Schools. 2017. CSS Dropdowns. https://www.w3schools.com/css/ css_dropdowns.asp

[55]

Thomas A. Walsh, Gregory M. Kapfhammer, and Phil McMinn. 2017. Automated Layout Failure Detection for Responsive Web Pages Without an Explicit Oracle. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2017). ACM, 192–202.

Digital Library

[56]

T. A. Walsh, P. McMinn, and G. M. Kapfhammer. 2015. Automatic Detection of Potential Layout Faults Following Changes to Responsive Web Pages (N). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 709–714.

[57]

Web Platform Tests. 2017. web-platform-tests dashboard, WPT/css/CSS2/floats. https://wptdashboard.appspot.com/css/ CSS2/floats

[58]

WebAIM. 2017. WAVE Web Accessibility Tool. http://wave.webaim. org/

[59]

Hadley Wickham. 2009. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, New York, New York, USA. http://ggplot2. org

Digital Library

[60]

Leland Wilkinson. 2005. The Grammar of Graphics (Statistics and Computing). Springer-Verlag New York, Inc., Secaucus, NJ, USA.

Digital Library

[61]

Qing Xie and Atif M. Memon. 2007. Designing and Comparing Automated Test Oracles for GUI-based Software Applications. ACM Trans. Softw. Eng. Methodol. 16, 1, Article 4 (Feb. 2007), 38 pages.

Digital Library

[62]

Brad Vander Zanden and Brad A. Myers. 1991. The Lapidary Graphical Interface Design Tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’91). ACM, 465–466.

Digital Library

Cited By

Chiou PWinn RAlotaibi AHalfond WRoychoudhury APaiva AAbreu RStorey M(2024)Automatically Detecting Reflow Accessibility Issues in Responsive Web PagesProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639229(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639229
Song SLee BChandra SBlincoe KTonella P(2023)Metamong: Detecting Render-Update Bugs in Web Browsers through FuzzingProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616336(1075-1087)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616336
Liu JChen YAtkinson EFeng YBodik R(2023)Conflict-Driven Synthesis for Layout EnginesProceedings of the ACM on Programming Languages10.1145/35912467:PLDI(638-659)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591246
Show More Cited By

Index Terms

Verifying that web pages have accessible layout
1. Software and its engineering
  1. Software notations and tools
    1. Context specific languages
      1. Graphical user interface languages
    2. Software maintenance tools

Recommendations

Verifying that web pages have accessible layout
PLDI '18

Usability and accessibility guidelines aim to make graphical user interfaces accessible to all users, by, say, requiring that text is sufficiently large, interactive controls are visible, and heading size corresponds to importance. These guidelines must ...
Modular verification of web page layout

Automated verification can ensure that a web page satisfies accessibility, usability, and design properties regardless of the end user's device, preferences, and assistive technologies. However, state-of-the-art verification tools for layout properties ...
Designing web accessibility

The majority of websites have not been designed to make them accessible to all the people for whom they are intended. Nowadays, eEurope initiative (European Commission-eEurope, 2005), wants the institutions to become aware of the necessity of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation

June 2018

825 pages

ISBN:9781450356985

DOI:10.1145/3192366

General Chair:
Jeffrey S. Foster
University of Maryland at College Park, USA
,
Program Chair:
Dan Grossman
University of Washington, USA

ACM SIGPLAN Notices Volume 53, Issue 4
PLDI '18
April 2018
834 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3296979
Editor:
Matthew Fluet
Rodchester Institude of Technology
Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

PLDI '18

Sponsor:

SIGPLAN

PLDI '18: ACM SIGPLAN Conference on Programming Language Design and Implementation

June 18 - 22, 2018

PA, Philadelphia, USA

Acceptance Rates

Overall Acceptance Rate 406 of 2,067 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
1,492
Total Downloads

Downloads (Last 12 months)271
Downloads (Last 6 weeks)28

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chiou PWinn RAlotaibi AHalfond WRoychoudhury APaiva AAbreu RStorey M(2024)Automatically Detecting Reflow Accessibility Issues in Responsive Web PagesProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639229(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639229
Song SLee BChandra SBlincoe KTonella P(2023)Metamong: Detecting Render-Update Bugs in Web Browsers through FuzzingProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616336(1075-1087)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616336
Liu JChen YAtkinson EFeng YBodik R(2023)Conflict-Driven Synthesis for Layout EnginesProceedings of the ACM on Programming Languages10.1145/35912467:PLDI(638-659)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591246
Chiou PAlotaibi AHalfond W(2023)BAGEL: An Approach to Automatically Detect Navigation-Based Web Accessibility Barriers for Keyboard UsersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580749(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580749
Yang XLiu WLin HLi ZQian FWang XLiu YXu T(2023)Visual-Aware Testing and Debugging for Web Performance OptimizationProceedings of the ACM Web Conference 202310.1145/3543507.3583323(2948-2959)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583323
Alotaibi AChiou PTawsif FHalfond W(2023)ScaleFix: An Automated Repair of UI Scaling Accessibility Issues in Android Applications2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58846.2023.00025(147-159)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICSME58846.2023.00025
Chiou PAlotaibi AHalfond WGrundy JPollock LPenta M(2023)Detecting Dialog-Related Keyboard Navigation Failures in Web ApplicationsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00120(1368-1380)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00120
Prazina IBecirovic SCogo EOkanovic V(2023)Methods for Automatic Web Page Layout Testing and Analysis: A ReviewIEEE Access10.1109/ACCESS.2023.324254911(13948-13964)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3242549
Altmajer CBłażewicz PSkublewska-Paszkowska M(2022)Analysis of the graphical user interface of the online store, taking into account the methods of universal designJournal of Computer Sciences Institute10.35784/jcsi.301025(315-322)Online publication date: 30-Dec-2022
https://doi.org/10.35784/jcsi.3010
Song SHur JKim SRogers PLee BDwyer MDamian DZeller A(2022)R2Z2Proceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510044(1818-1829)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510044
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten