demonstration

MDroid+: a mutation testing framework for android

Authors:

Michele Tufano,

Carlos Bernal-Cárdenas,

Mario Linares-Vásquez,

Gabriele Bavota,

Christopher Vendome,

Massimiliano Di Penta,

Denys PoshyvanykAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Pages 33 - 36

https://doi.org/10.1145/3183440.3183492

Published: 27 May 2018 Publication History

Abstract

Mutation testing has shown great promise in assessing the effectiveness of test suites while exhibiting additional applications to test-case generation, selection, and prioritization. Traditional mutation testing typically utilizes a set of simple language specific source code transformations, called operators, to introduce faults. However, empirical studies have shown that for mutation testing to be most effective, these simple operators must be augmented with operators specific to the domain of the software under test. One challenging software domain for the application of mutation testing is that of mobile apps. While mobile devices and accompanying apps have become a mainstay of modern computing, the frameworks and patterns utilized in their development make testing and verification particularly difficult. As a step toward helping to measure and ensure the effectiveness of mobile testing practices, we introduce MDroid+, an automated framework for mutation testing of Android apps. MDroid+ includes 38 mutation operators from ten empirically derived types of Android faults and has been applied to generate over 8,000 mutants for more than 50 apps.

Video URL: https://youtu.be/yzE5_-zN5GA

References

[1]

Google's Android just beat Microsoft Windows as most popular OS http://fortune.com/2017/04/03/google-android-microsoft-os/, 2017.

[2]

J. H. Andrews, L. C. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments? In ICSE'05, pages 402--411, 2005.

Digital Library

[3]

J. H. Andrews, L. C. Briand, Y. Labiche, and A. S. Namin. Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Trans. Software Eng., 32(8):608--624.

Digital Library

[4]

Apple. App store. https://itunes.apple.com/us/genre/ios/id36?mt=8, 2017.

[5]

G. Bavota, M. Linares-Vásquez, C. Bernal-Cárdenas, M. Di Penta, R. Oliveto, and D. Poshyvanyk. The impact of API change- and fault-proneness on the user ratings of Android apps. IEEE Trans. Software Eng., 41(5):384--407, 2015.

Digital Library

[6]

H. Coles. Pit. http://pitest.org/, 2017.

[7]

M. Daran and P. Thévenod-Fosse. Software error analysis: A real case study involving real faults and mutations. In ISSTA'96, pages 158--171, 1996.

Digital Library

[8]

R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on test data selection: Help for the practicing programmer. Computer, 11(4):34--41, April 1978.

Digital Library

[9]

L. Deng, N. Mirzaei, P. Ammann, and J. Offutt. Towards mutation analysis of Android apps. In ICSTW'15, pages 1--10, April 2015.

[10]

Google. Google play. https://play.google.com/store, 2017.

[11]

R. G. Hamlet. Testing programs with the aid of a compiler. IEEE Trans. Software Eng., 3(4):279--290, July 1977.

Digital Library

[12]

D. Han, C. Zhang, X. Fan, A. Hindle, K. Wong, and E. Stroulia. Understanding Android fragmentation with topic analysis of vendor-specific bugs. In WCRE'12, pages 83--92, 2012.

Digital Library

[13]

S. Holm. A simple sequentially rejective Bonferroni test procedure. Scandinavian Journal on Statistics, 6:65--70, 1979.

[14]

R. Just. The Major mutation framework: Efficient and scalable mutation analysis for java. In International Symposium on Software Testing and Analysis, ISSTA'14.

Digital Library

[15]

R. Just, D. Jalali, L. Inozemtseva, M. D. Ernst, R. Holmes, and G. Fraser. Are mutants a valid substitute for real faults in software testing? In ESEC/FSE'14.

Digital Library

[16]

M. Linares-Vásquez, G. Bavota, C. Bernal-Cárdenas, M. Di Penta, R. Oliveto, and D. Poshyvanyk. API change and fault proneness: A threat to the success of Android apps. In ESEC/FSE'13, pages 477--487, 2013.

Digital Library

[17]

M. Linares-Vásquez, G. Bavota, M. Tufano, K. Moran, M. Di Penta, C. Vendome, C. Bernal-Cárdenas, and D. Poshyvanyk. Enabling mutation testing for android apps. In ESEC/FSE'17, pages 233--244, 2017.

Digital Library

[18]

M. Linares-Vásquez, G. Bavota, M. Tufano, K. Moran, M. Di Penta, C. Vendome, C. Bernal-Cárdenas, and D. Poshyvanyk. Online appendix for "enabling mutation testing for android apps". http://android-mutation.com, 2017.

[19]

M. Linares-Vásquez, C. Bernal-Cárdenas, K. Moran, and D. Poshyvanyk. How do developers test android applications? In ICSME'17, pages 613--622, Sept 2017.

[20]

Q. Luo, K. Moran, and D. Poshyvanyk. A large-scale empirical comparison of static and dynamic test case prioritization techniques. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, pages 559--570, New York, NY, USA, 2016. ACM.

Digital Library

[21]

W. Martin, F. Sarro, and M. Harman. Causal impact analysis for app releases in google play. In ESEC/FSE'16, pages 435--446, 2016.

Digital Library

[22]

M. B. Miles, A. M. Huberman, and J. Saldaña. Qualitative Data Analysis: A Methods Sourcebook. SAGE Publications, Inc, 3rd edition, Apr 2013.

[23]

A.J. Offutt and R. H. Untch. Mutation 2000: Uniting the Orthogonal, pages 34--44. Springer US, Boston, MA, 2001.

[24]

S. Roy Choudhary, A. Gorla, and A. Orso. Automated test input generation for android: Are we there yet? (E). In ASE '15, pages 429--440, 2015.

Digital Library

Cited By

Ami AAhmed SRedoy RCooper NKafle KMoran KPoshyvanyk DNadkarni AChandra SBlincoe KTonella P(2023)MASC: A Tool for Mutation-Based Evaluation of Static Crypto-API Misuse DetectorsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3613099(2162-2166)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3613099
Ojdanic MPapadakis MHarman MChandra SBlincoe KTonella P(2023)Keeping Mutation Test Suites Consistent and Relevant with Long-Standing MutantsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3613089(2067-2071)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3613089
Du HPalepu VJones JJust RFraser G(2023)To Kill a Mutant: An Empirical Study of Mutation Testing KillsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598090(715-726)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598090
Show More Cited By

Index Terms

MDroid+: a mutation testing framework for android
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation

Recommendations

µDroid: an energy-aware mutation testing framework for Android
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering

The rising popularity of mobile apps deployed on battery-constrained devices underlines the need for effectively evaluating their energy properties. However, currently there is a lack of testing tools for evaluating the energy properties of apps. As a ...
Enabling mutation testing for Android apps
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering

Mutation testing has been widely used to assess the fault-detection effectiveness of a test suite, as well as to guide test case generation or prioritization. Empirical studies have shown that, while mutants are generally representative of real faults, ...
DroidMutator: an effective mutation analysis tool for Android applications
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

With the rapid growth of Android devices, techniques that ensure high quality of mobile applications (i.e., apps) are receiving more and more attention. It is well-accepted that mutation analysis is an effective approach to simulate and locate realistic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

May 2018

231 pages

ISBN:9781450356633

DOI:10.1145/3183440

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Copyright © 2018 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
299
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ami AAhmed SRedoy RCooper NKafle KMoran KPoshyvanyk DNadkarni AChandra SBlincoe KTonella P(2023)MASC: A Tool for Mutation-Based Evaluation of Static Crypto-API Misuse DetectorsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3613099(2162-2166)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3613099
Ojdanic MPapadakis MHarman MChandra SBlincoe KTonella P(2023)Keeping Mutation Test Suites Consistent and Relevant with Long-Standing MutantsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3613089(2067-2071)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3613089
Du HPalepu VJones JJust RFraser G(2023)To Kill a Mutant: An Empirical Study of Mutation Testing KillsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598090(715-726)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598090
Clerissi DDi Rocco JDi Ruscio DDi Sipio CIhirwe FMariani LMicucci DRossi MRubei R(2023)Supporting Early-Safety Analysis of IoT Systems by Exploiting Testing Techniques2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00089(520-529)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00089
Salihu IUsman AEke NIbrahim RKusharki M(2023)Mutation Testing Techniques for Android Applications: A Comparative Study2023 2nd International Conference on Multidisciplinary Engineering and Applied Science (ICMEAS)10.1109/ICMEAS58693.2023.10429866(1-5)Online publication date: 1-Nov-2023
https://doi.org/10.1109/ICMEAS58693.2023.10429866
Jie LAhmad JHassan R(2023)Code annotation mutation operators for android application27TH INTERNATIONAL MEETING OF THERMOPHYSICS 202210.1063/5.0164487(030018)Online publication date: 2023
https://doi.org/10.1063/5.0164487
Leotta MPaparella DRicca F(2023)Mutta: a novel tool for E2E web mutation testingSoftware Quality Journal10.1007/s11219-023-09616-632:1(5-26)Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1007/s11219-023-09616-6
Su TFan LChen SLiu YXu LPu GSu Z(2022)Why My App Crashes? Understanding and Benchmarking Framework-Specific Exceptions of Android AppsIEEE Transactions on Software Engineering10.1109/TSE.2020.301343848:4(1115-1137)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TSE.2020.3013438
Lu YShao KSun WSun M(2022)RGChaser: A RL-guided Fuzz and Mutation Testing Framework for Deep Learning Systems2022 9th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA56465.2022.00012(12-23)Online publication date: Aug-2022
https://doi.org/10.1109/DSA56465.2022.00012
Lu YShao KSun WSun M(2022)MTUL: Towards Mutation Testing of Unsupervised Learning SystemsDependable Software Engineering. Theories, Tools, and Applications10.1007/978-3-031-21213-0_2(22-40)Online publication date: 24-Oct-2022
https://dl.acm.org/doi/10.1007/978-3-031-21213-0_2
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents