On the costs of applying logic-based criteria to mobile applications: An empirical analysis of predicates in real-world Objective-C and Swift applications
Article No.: 4, Pages 1 - 9
Abstract
The proliferation of mobile devices has given rise to an increasing demand for software that is well-suited to this particular environment. However, ensuring the quality of mobile applications is challenging. Much of the overall complexity of mobile applications stems from logic expressions, i.e., predicates, which appear in control-flow statements (e.g., if, if--else, while, and do--while) and define much of the behavior of software. Thus, testing predicates is key to ensuring the quality of mobile applications. We argue that an apt way to test predicates is by leveraging well-established logic-based criteria. Many logic-based criteria have been devised, e.g., the active clause coverage (ACC) and modified condition/decision coverage (MCDC). Given that ACC/MCDC are considered expensive, we set out to examine the cost of applying these criteria to mobile applications. We probed into a basic, but relevant, proxy for cost: the complexity of predicates, i.e., number of clauses in predicates. We examined 35 open-source mobile applications implemented in Objective-C and Swift ranging from 129 to 58,140 lines of code with a total of 19,345 predicates. We looked at the frequency and percentage of predicates. We also analyzed the relationship between overall measures of size and the frequency of predicates. We found that, although about 99% of the predicates in mobile application have at most three clauses, there is a significant positive linear correlation between overall measures of size and the number of predicates with four or more clauses. We conclude that mobile applications do not have many multi-clause predicates and hence sophisticated logic-based criteria are needed on only a small portion of the predicates.
References
[1]
P. Ammann and J. Offutt. 2008. Introduction to Software Testing. Cambridge University Press. 344 pages.
[2]
P. Ammann, J. Offutt, and H. Huang. 2003. Coverage Criteria for Logical Expressions. In 14th International Symposium on Software Reliability Engineering (ISSRE).99--107.
[3]
Bruce J. Chalmer. 1986. Understanding Statistics. CRC Press. 448 pages.
[4]
J. J. Chilenski. 2001. An Investigation of Three Forms of the Modified Condition Decision Coverage (MCDC) Criterion. Technical Report DOT/FAA/AR-01/18. FAA Tech Center Report.
[5]
J. J. Chilenski and S. P. Miller. 1994. Applicability of Modified Condition/Decision Coverage to Software Testing. Software Engineering Journal 9, 5 (1994), 193--200.
[6]
J. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge Academic. 567 pages.
[7]
V. H. S. Durelli, J. Offutt, N. Li, M. E. Delamaro, J. Guo, Z. Shi, and X. Ai. 2016. What to Expect of Predicates: An Empirical Analysis of Predicates in Real World Programs. Journal of Systems and Software 113 (2016), 324--336.
[8]
J. Miles and M. Shevlin. 2000. Applying Regression and Correlation: A Guide for Students and Researchers. Sage Publications. 274 pages.
[9]
J. Miller. 2004. Statistical Significance Testing - A Panacea for Software Technology Experiments? Journal of Systems and Software 73, 2 (2004), 183--192.
[10]
Open-Source iOS Apps. 2017. https://github.com/dkhamsing/open-source-ios-apps. (2017). Accessed: May 20, 2017.
[11]
T. Parr. 2013. The Definitive ANTLR 4 Reference (2nd ed.). Pragmatic Bookshelf. 328 pages.
[12]
RTCA-DO-178B. 1992. Software Considerations in Airborne Systems and Equipment Certification. (December 1992).
[13]
F. Q. B. Silva, M. Suassuna, A. C. C. França, A. M. Grubb, T. B. Gouveia, C. V. F. Monteiro, and I. E. dos Santos. 2014. Replication of Empirical Studies in Software Engineering Research: A Systematic Mapping Study. Empirical Software Engineering 19, 3 (2014), 501--557.
[14]
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. 2012. Experimentation in Software Engineering. Springer. 236 pages.
[15]
S. Zein, N. Salleh, and J. Grundy. 2016. A Systematic Mapping Study of Mobile Application Testing Techniques. Journal of Systems and Software 117 (2016), 334--356.
Index Terms
- On the costs of applying logic-based criteria to mobile applications: An empirical analysis of predicates in real-world Objective-C and Swift applications
Recommendations
What to expect of predicates
We analyzed 63 open source, non-safety-critical Java programs ranging from 423 to 629,114 lines of code, with a total of 400,811 predicates.We also analyzed three railway safety-critical programs implemented in C/C++ whose sizes range from 42,380 to 65,...
Comments
Information & Contributors
Information
Published In
September 2017
100 pages
ISBN:9781450353021
DOI:10.1145/3128473
Copyright © 2017 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- SBC: Sociedade Brasileira de Computação
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 September 2017
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
SAST '17
SAST '17: 2nd Brazilian Symposium on Systematic and Automated Software Testing
September 18 - 19, 2017
Fortaleza, Brazil
Acceptance Rates
SAST '17 Paper Acceptance Rate 11 of 16 submissions, 69%;
Overall Acceptance Rate 45 of 92 submissions, 49%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 87Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 31 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in