skip to main content
10.1145/3128473.3128477acmotherconferencesArticle/Chapter ViewAbstractPublication PagessastConference Proceedingsconference-collections
research-article

On the costs of applying logic-based criteria to mobile applications: An empirical analysis of predicates in real-world Objective-C and Swift applications

Published: 18 September 2017 Publication History

Abstract

The proliferation of mobile devices has given rise to an increasing demand for software that is well-suited to this particular environment. However, ensuring the quality of mobile applications is challenging. Much of the overall complexity of mobile applications stems from logic expressions, i.e., predicates, which appear in control-flow statements (e.g., if, if--else, while, and do--while) and define much of the behavior of software. Thus, testing predicates is key to ensuring the quality of mobile applications. We argue that an apt way to test predicates is by leveraging well-established logic-based criteria. Many logic-based criteria have been devised, e.g., the active clause coverage (ACC) and modified condition/decision coverage (MCDC). Given that ACC/MCDC are considered expensive, we set out to examine the cost of applying these criteria to mobile applications. We probed into a basic, but relevant, proxy for cost: the complexity of predicates, i.e., number of clauses in predicates. We examined 35 open-source mobile applications implemented in Objective-C and Swift ranging from 129 to 58,140 lines of code with a total of 19,345 predicates. We looked at the frequency and percentage of predicates. We also analyzed the relationship between overall measures of size and the frequency of predicates. We found that, although about 99% of the predicates in mobile application have at most three clauses, there is a significant positive linear correlation between overall measures of size and the number of predicates with four or more clauses. We conclude that mobile applications do not have many multi-clause predicates and hence sophisticated logic-based criteria are needed on only a small portion of the predicates.

References

[1]
P. Ammann and J. Offutt. 2008. Introduction to Software Testing. Cambridge University Press. 344 pages.
[2]
P. Ammann, J. Offutt, and H. Huang. 2003. Coverage Criteria for Logical Expressions. In 14th International Symposium on Software Reliability Engineering (ISSRE).99--107.
[3]
Bruce J. Chalmer. 1986. Understanding Statistics. CRC Press. 448 pages.
[4]
J. J. Chilenski. 2001. An Investigation of Three Forms of the Modified Condition Decision Coverage (MCDC) Criterion. Technical Report DOT/FAA/AR-01/18. FAA Tech Center Report.
[5]
J. J. Chilenski and S. P. Miller. 1994. Applicability of Modified Condition/Decision Coverage to Software Testing. Software Engineering Journal 9, 5 (1994), 193--200.
[6]
J. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge Academic. 567 pages.
[7]
V. H. S. Durelli, J. Offutt, N. Li, M. E. Delamaro, J. Guo, Z. Shi, and X. Ai. 2016. What to Expect of Predicates: An Empirical Analysis of Predicates in Real World Programs. Journal of Systems and Software 113 (2016), 324--336.
[8]
J. Miles and M. Shevlin. 2000. Applying Regression and Correlation: A Guide for Students and Researchers. Sage Publications. 274 pages.
[9]
J. Miller. 2004. Statistical Significance Testing - A Panacea for Software Technology Experiments? Journal of Systems and Software 73, 2 (2004), 183--192.
[10]
Open-Source iOS Apps. 2017. https://github.com/dkhamsing/open-source-ios-apps. (2017). Accessed: May 20, 2017.
[11]
T. Parr. 2013. The Definitive ANTLR 4 Reference (2nd ed.). Pragmatic Bookshelf. 328 pages.
[12]
RTCA-DO-178B. 1992. Software Considerations in Airborne Systems and Equipment Certification. (December 1992).
[13]
F. Q. B. Silva, M. Suassuna, A. C. C. França, A. M. Grubb, T. B. Gouveia, C. V. F. Monteiro, and I. E. dos Santos. 2014. Replication of Empirical Studies in Software Engineering Research: A Systematic Mapping Study. Empirical Software Engineering 19, 3 (2014), 501--557.
[14]
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén. 2012. Experimentation in Software Engineering. Springer. 236 pages.
[15]
S. Zein, N. Salleh, and J. Grundy. 2016. A Systematic Mapping Study of Mobile Application Testing Techniques. Journal of Systems and Software 117 (2016), 334--356.

Index Terms

  1. On the costs of applying logic-based criteria to mobile applications: An empirical analysis of predicates in real-world Objective-C and Swift applications

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    SAST '17: Proceedings of the 2nd Brazilian Symposium on Systematic and Automated Software Testing
    September 2017
    100 pages
    ISBN:9781450353021
    DOI:10.1145/3128473
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    • SBC: Sociedade Brasileira de Computação

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 September 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Active clause coverage (ACC) criteria
    2. Logic-based test criteria
    3. Modified condition-decision coverage
    4. Replication study

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    SAST '17

    Acceptance Rates

    SAST '17 Paper Acceptance Rate 11 of 16 submissions, 69%;
    Overall Acceptance Rate 45 of 92 submissions, 49%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 87
      Total Downloads
    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 31 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media