research-article

API usability peer reviews: a method for evaluating the usability of application programming interfaces

Authors:

Dieter ZirklerAuthors Info & Claims

CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 2327 - 2336

https://doi.org/10.1145/1753326.1753677

Published: 10 April 2010 Publication History

Abstract

We describe a usability inspection method to evaluate Application Programming Interfaces (APIs). We found the method useful as it identified usability defects in Microsoft's .NET Framework, of which 59% were new and 21% were fixed. Based on a comparison of usability defects identified between API usability peer reviews and API usability tests, API usability tests were found to expose design issues related to actually using an API whereas API usability peer reviews were found to expose the design rationale of an API. We reflect on the efficiency and productivity of each method: each API usability test is equivalent to 16 API usability peer reviews with the former having a 2.5x productivity advantage. We discuss how API usability peer reviews can be used in conjunction with API usability tests to increase usability coverage on APIs.

References

[1]

Ackerman, A.F., Buchwald, L.S., and Lewski, F.H. Software inspections: An effective verification process. IEEE Software 6, 3 (May 1989), 31--36.

Digital Library

[2]

Bell, B. Using programming walkthroughs to design a visual language. Ph.D. dissertation, University of Colorado, 1992.

Digital Library

[3]

Bell, B., Citrin, W., Lewis, C., Rieman, J., Wilde, N., and Zorn, B. The programming walkthrough: A structured method for assessing the writability of programming languages. Software Practice and Experience 24, 1 (January 1994), 1--25.

Digital Library

[4]

Bloch, J. How to write a good API and why it matters. Keynote address for LCSD workshop at OOPSLA, 2005. http://lcsd05.cs.tamu.edu/#keynote.

Digital Library

[5]

Cuomo, D.L. and Bowen, C.D. Understanding usability issues addressed by three user-system interface evaluation techniques. Interacting with Computers 6, 1 (1994), 86--108.

[6]

Cwalina, K. and Abrams, B. Framework design guidelines. Addison-Wesley, 2005.

[7]

Daughtry, J.M., Farooq, U., Stylos, J., and Myers, B.A. API usability: CHI'09 Special Interest Group. Proc. CHI 2009, ACM Press (2009), 2771--2774.

Digital Library

[8]

Daughtry, J.M., Farooq, U., Myers, B.A, and Stylos, J. API usability: Report on Special Interest Group at CHI. Software Engineering Notes 34, 4 (July 2009), 27--29.

Digital Library

[9]

Desurvire, H.W. Faster, Cheaper!! Are Usability Inspection Methods as Effective as Empirical Testing? In Nielsen, J. and Mack, R.L. Usability inspection methods (Ed). John Wiley & Sons, 1994, 173--201.

Digital Library

[10]

Fagan, M.E. Design and code inspection to reduce errors in program development. IBM Systems Journal 15, 3 (1976), 182--211.

Digital Library

[11]

Fagan, M.E. Advances in software inspection. IEEE Transactions on Software Engineering 12, 7 (July, 1986), 744--751.

Digital Library

[12]

Feature Crews: How Microsoft Does It. CodePlex: Open Source Community, 2007. http://www.codeplex.com/ BranchingGuidance/Wiki/View.aspx?title=Feature%20Crews%3a%20How%20Microsoft%20Does%20It&referringTitle=Home.

[13]

Freedman, D. and Weinberg, G.M. Handbook of Walkthroughs, Inspections, and Technical Reviews: Evaluating Programs, Projects, and Products. New York: Dorset House, 1990.

Digital Library

[14]

Google Maps API. http://code.google.com/apis/maps/.

[15]

Gray, W.D. and Salzman, M.C. Damaged merchandise? A review of experiments that compare usability evaluation methods. Human Computer Interaction 13, 3 (1998) 203--261.

Digital Library

[16]

Green, T.R.G. and Petre, M. Usability analysis of visual programming environments: A 'cognitive dimensions' framework. Journal of Visual Languages & Computing 7, 2 (1996), 131--174.

[17]

Greenberg, S. and Buxton, B. Usability evaluation considered harmful (some of the time). Proc. CHI 2008, ACM Press (2008), 111--120.

Digital Library

[18]

Hammond, N., Hinton, G., Barnard, P., MacLean, A., Long, J., and Whitefield, A. Evaluating the interface of a document processor: A comparison of expert judgment and user observation. Proc. of IFIP INTERACT 1994, Elsevier Science Publishers (1994), 725--729.

[19]

Henning, M. API Design Matters. Communications of the ACM 52, 5 (May 2009), 46--56.

Digital Library

[20]

iPhone Developer Program. http://developer.apple.com/iPhone/program.

[21]

Jeffries, R. and Desurvire, H. Usability testing vs. heuristic evaluation: Was there a contest? ACM SIGCHI Bulletin 24, 4 (October 1992), 39--41.

Digital Library

[22]

Jeffries, R., Miller, J.R., Wharton, C., and Uyeda, K.M. User Interface Evaluation in the Real World: A Comparison of Four Techniques. Proc. CHI 1991, ACM Press (1991), 119--124.

Digital Library

[23]

John, B.E. and Marks, S.J. Tracking the effectiveness of usability evaluation methods. Behaviour & Information Technology 16, 4/5 (1997), 188--202.

[24]

Kahn, M.J. and Prail, A. Formal usability inspections. In Nielsen, J. and Mack, R.L. Usability inspection methods (Ed). John Wiley & Sons, 1994, 141--171.

Digital Library

[25]

Karat, C.-M., Campbell, R., and Fiegel, T. Comparison of empirical testing and walkthrough methods in user interface evaluation. Proc. CHI 1992, ACM Press (1992), 397--404.

Digital Library

[26]

Lewis, C., Polson, P.G., Wharton, C., and Rieman, J. Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces. Proc. CHI 1990, ACM Press (1990), 235--242.

Digital Library

[27]

Mack, R.L. and Nielsen, J. Executive summary. In Nielsen, J. and Mack, R.L. Usability inspection methods (Ed). John Wiley & Sons, 1994, 1--23.

Digital Library

[28]

Nielsen, J. and Mack, R.L. Usability inspection methods (Ed). John Wiley & Sons, 1994.

Digital Library

[29]

Nielsen, J. and Phillips, V. Estimating the relative usability of two interfaces. Proc. CHI 1993, ACM Press (1993), 214--221.

Digital Library

[30]

Olson, G.M., and Moran, T.P. Commentary on "Damaged Merchandise". Human Computer Interaction 13, 3 (1998), 263--323.

Digital Library

[31]

Rieman, J., Franzke, M., and Redmiles, D. Usability evaluation with the cognitive walkthrough. Proc. CHI 1995, ACM Press (1995), 387--388.

Digital Library

[32]

Russell, G.W. Experience with inspection in ultralarge-scale developments. IEEE Software 8, 1 (January 1991), 25--31.

Digital Library

[33]

Savage, P. User interface evaluation in an iterative design process: A comparison of three techniques. Proc. CHI 1996, ACM Press (1996), 307--308.

Digital Library

[34]

Schwaber, K. and Beedle, M. Agile software development with SCRUM. Prentice Hall, 2002.

Digital Library

[35]

Stevens, S.M. Intelligent interactive video simulation of a code inspection. Communications of the ACM 32, 7 (July, 1989), 832--843.

Digital Library

[36]

Stylos, J. and Clarke, S. Usability implications for requiring parameters in objects' constructors. Proc. ICSE 2007, ACM Press (2007), 529--539.

Digital Library

[37]

Team Foundation Server. http://msdn.microsoft.com/en-us/teamsystem/dd408382.aspx.

[38]

Visual Studio IntelliSense. http://msdn.microsoft.com/en-us/library/hcw1s69b(VS.71).aspx.

[39]

Vygotsky, L.S. Mind in society: The development of higher psychological processes. Harvard University Press, 1978.

[40]

Weller, E.F. Lessons from three years of inspection data. IEEE Software 10, 5 (September, 1993), 38--45.

Digital Library

[41]

Wharton, C., Rieman, J., Lewis, C. and Polson, P. The cognitive walkthrough method: A practitioner's guide. In J. Nielsen & R.L. Mack, Usability inspection methods, John Wiley & Sons, 1994 (pp. 105--140).

Digital Library

[42]

Wiegers, K.E. Peer reviews in software: A practical guide. Addison-Wesley, 2002.

Digital Library

Cited By

Ahmad MGeewax JMacvean AKarger DMa KRoychoudhury APaiva AAbreu RStorey MAniche MNagappan N(2024)API Governance at ScaleProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639713(430-440)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639477.3639713
Ahmad MMacvean AKarger DMa K(2024)AI-Enhanced API Design: A New Paradigm in Usability and EfficiencyExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650803(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650803
de Ryckel XSluÿters AVanderdonckt J(2022)SnappView, a Software Development Kit for Supporting End-user Mobile Interface ReviewProceedings of the ACM on Human-Computer Interaction10.1145/35345276:EICS(1-38)Online publication date: 17-Jun-2022
https://dl.acm.org/doi/10.1145/3534527
Show More Cited By

Index Terms

API usability peer reviews: a method for evaluating the usability of application programming interfaces
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

API peer reviews: a method for evaluating usability of application programming interfaces
CSCW '10: Proceedings of the 2010 ACM conference on Computer supported cooperative work

API usability tests in the lab are time and resource intensive, thus allowing a relatively small percentage of the API namespace to be evaluated. We describe a group-based usability inspection method -- API Peer Reviews -- to evaluate API usability. ...
A systematic mapping study of API usability evaluation methods
Abstract
An Application Programming Interface (API) provides a programmatic interface to a software component that is often offered publicly and may be used by programmers who are not the API’s original designers. APIs play a key role in software reuse. ...
Some structural measures of API usability

In this age of collaborative software development, the importance of usable APIs is well recognized. There already exists a rich body of literature that addresses issues ranging from how to design usable APIs to assessing qualitatively the usability of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2010

2690 pages

ISBN:9781605589299

DOI:10.1145/1753326

General Chair:
Elizabeth Mynatt
Georgia Institute of Technology
,
Program Chairs:
Geraldine Fitzpatrick
Vienna University of Technology
,
Scott Hudson
Carnegie Mellon University
,
Keith Edwards
Georgia Tech
,
Tom Rodden
University of Nottingham

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '10

Sponsor:

SIGCHI

CHI '10: CHI Conference on Human Factors in Computing Systems

April 10 - 15, 2010

Georgia, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
1,147
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)7

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ahmad MGeewax JMacvean AKarger DMa KRoychoudhury APaiva AAbreu RStorey MAniche MNagappan N(2024)API Governance at ScaleProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639713(430-440)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639477.3639713
Ahmad MMacvean AKarger DMa K(2024)AI-Enhanced API Design: A New Paradigm in Usability and EfficiencyExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650803(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650803
de Ryckel XSluÿters AVanderdonckt J(2022)SnappView, a Software Development Kit for Supporting End-user Mobile Interface ReviewProceedings of the ACM on Human-Computer Interaction10.1145/35345276:EICS(1-38)Online publication date: 17-Jun-2022
https://dl.acm.org/doi/10.1145/3534527
Bastos Jde Mello RGarcia ACavalcante EDantas FBatista T(2020)ColloquyProceedings of the XXXIV Brazilian Symposium on Software Engineering10.1145/3422392.3422468(514-519)Online publication date: 21-Oct-2020
https://dl.acm.org/doi/10.1145/3422392.3422468
Zhang THartmann BKim MGlassman EBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Enabling Data-Driven API Design with Community Usage DataProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376382(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376382
Nam DZimmermann TLawall JMarinov D(2019)API design implications of boilerplate client codeProceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2019.00153(1253-1255)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1109/ASE.2019.00153
Nam DHorvath AMacvean AMyers BVasilescu BZimmermann TLawall JMarinov D(2019)MARBLEProceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2019.00063(615-627)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1109/ASE.2019.00063
Zghidi AHammouda IHnich BChaudron MCrnkovic IChechik MHarman M(2018)Towards a formal API assessmentProceedings of the 40th International Conference on Software Engineering: Companion Proceeedings10.1145/3183440.3195026(398-399)Online publication date: 27-May-2018
https://dl.acm.org/doi/10.1145/3183440.3195026
Murphy LKery MAlliyu OMacvean AMyers B(2018)API Designers in the Field: Design Practices and Challenges for Creating Usable APIs2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VLHCC.2018.8506523(249-258)Online publication date: Oct-2018
https://doi.org/10.1109/VLHCC.2018.8506523
Kwon HAhn JChoi SSiewierski JKosko PSzydelko P(2018)An Experience Report of the API Evolution and Maintenance for Software Platforms2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME.2018.00034(587-590)Online publication date: Sep-2018
https://doi.org/10.1109/ICSME.2018.00034
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents