Towards a formal API assessment

Assessing the quality of an API is important in many different aspects: First, it can assist developers in deciding which API to use when they are faced with a list of potential APIs to choose from, by comparing the benefits and drawbacks of each option [1]; we refer to this as the API selection problem. Second, it can help guide the design process and expose problem areas in early stages of API design, even before implementing the actual API [2]; we refer to this as the API design problem. In order to assess the quality of an API, various evaluation methods have been used: some are based on empirical laboratory studies, gathering feedback from API users; others are based on inspection methods where experts evaluate the quality of an API based on a list of design guidelines [3] [4] such as Nielsen's heuristics and the cognitive dimensions framework [2] [5]. In this paper, we are particularly interested in extending Steven Clarke's approach of measuring API usability based on the cognitive dimensions framework [5]. The usability of an API is assessed by comparing the API (what it actually offers) with the profiles of its potential users (what they expect out of it).