research-article

Mining Android API Usage to Generate Unit Test Cases for Pinpointing Compatibility Issues

Authors:

Li LiAuthors Info & Claims

ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

Article No.: 70, Pages 1 - 13

https://doi.org/10.1145/3551349.3561151

Published: 05 January 2023 Publication History

Abstract

Despite being one of the largest and most popular projects, the official Android framework has only provided test cases for less than 30% of its APIs. Such a poor test case coverage rate has led to many compatibility issues that can cause apps to crash at runtime on specific Android devices, resulting in poor user experiences for both apps and the Android ecosystem. To mitigate this impact, various approaches have been proposed to automatically detect such compatibility issues. Unfortunately, these approaches have only focused on detecting signature-induced compatibility issues (i.e., a certain API does not exist in certain Android versions), leaving other equally important types of compatibility issues unresolved. In this work, we propose a novel prototype tool, JUnitTestGen, to fill this gap by mining existing Android API usage to generate unit test cases. After locating Android API usage in given real-world Android apps, JUnitTestGen performs inter-procedural backward data-flow analysis to generate a minimal executable code snippet (i.e., test case). Experimental results on thousands of real-world Android apps show that JUnitTestGen is effective in generating valid unit test cases for Android APIs. We show that these generated test cases are indeed helpful for pinpointing compatibility issues, including ones involving semantic code changes.

References

[1]

24 April 2022. Kex. https://github.com/vorpal-research/kex/tree/sbst-contest. Online; accessed 24 April 2022.

[2]

Gabriele Bavota, Mario Linares-Vasquez, Carlos Eduardo Bernal-Cardenas, Massimiliano Di Penta, Rocco Oliveto, and Denys Poshyvanyk. 2014. The impact of api change-and fault-proneness on the user ratings of android apps. IEEE Transactions on Software Engineering 41, 4 (2014), 384–407.

Digital Library

[3]

Eric Bodden. 2012. Inter-procedural data-flow analysis with ifds/ide and soot. In Proceedings of the ACM SIGPLAN International Workshop on State of the Art in Java Program analysis. 3–8.

Digital Library

[4]

Haipeng Cai, Ziyi Zhang, Li Li, and Xiaoqin Fu. 2019. A large-scale study of application incompatibilities in Android. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 216–227.

Digital Library

[5]

Danny Dig and Ralph Johnson. 2005. The role of refactorings in API evolution. In 21st IEEE International Conference on Software Maintenance (ICSM’05). IEEE, 389–398.

Digital Library

[6]

Zhen Dong, Abhishek Tiwari, Xiao Liang Yu, and Abhik Roychoudhury. 2021. Flaky test detection in Android via event order exploration. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 367–378.

Digital Library

[7]

Gordon Fraser and Andrea Arcuri. 2011. Evolutionary generation of whole test suites. In 2011 11th International Conference on Quality Software. IEEE, 31–40.

Digital Library

[8]

Gordon Fraser and Andrea Arcuri. 2011. Evosuite: automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering. 416–419.

Digital Library

[9]

Google. 2021. LauncherApps. https://developer.android.com/reference/android/content/pm/LauncherApps#hasShortcutHostPermission(). Online; accessed 27 January 2021.

[10]

Hyung Kil Ham and Young Bom Park. 2011. Mobile application compatibility test system design for android fragmentation. In International Conference on Advanced Software Engineering and Its Applications. Springer, 314–320.

[11]

Huaxun Huang, Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2018. Understanding and detecting callback compatibility issues for android applications. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. 532–542.

Digital Library

[12]

Lingxiao Jiang and Zhendong Su. 2009. Automatic mining of functionally equivalent code fragments via random testing. In Proceedings of the eighteenth international symposium on Software testing and analysis. 81–92.

Digital Library

[13]

Muhammad Kamran, Junaid Rashid, and Muhammad Wasif Nisar. 2016. Android fragmentation classification, causes, problems and solutions. International Journal of Computer Science and Information Security 14, 9(2016), 992.

[14]

Puneet Kapur, Brad Cossette, and Robert J Walker. 2010. Refactoring references for library migration. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications. 726–738.

Digital Library

[15]

Maria Kechagia, Xavier Devroey, Annibale Panichella, Georgios Gousios, and Arie van Deursen. 2019. Effective and efficient API misuse detection via exception propagation and search-based testing. In Proceedings of the 28th ACM SIGSOFT international symposium on software testing and analysis. 192–203.

Digital Library

[16]

Maxime Lamothe, Weiyi Shang, and Tse-Hsun Peter Chen. 2020. A3: Assisting Android API migrations using code examples. IEEE Transactions on Software Engineering(2020).

[17]

Ondřej Lhoták and Laurie Hendren. 2003. Scaling Java points-to analysis using S park. In International Conference on Compiler Construction. Springer, 153–169.

Digital Library

[18]

Li Li, Tegawendé F Bissyandé, and Jacques Klein. 2018. MoonlightBox: Mining Android API Histories for Uncovering Release-time Inconsistencies. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE 2018).

[19]

Li Li, Tegawendé F Bissyandé, Yves Le Traon, and Jacques Klein. 2016. Accessing inaccessible android apis: An empirical study. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 411–422.

[20]

Li Li, Tegawendé F Bissyandé, Damien Octeau, and Jacques Klein. 2016. Droidra: Taming reflection to support whole-program analysis of android apps. In Proceedings of the 25th International Symposium on Software Testing and Analysis. 318–329.

Digital Library

[21]

Li Li, Tegawendé F Bissyandé, Mike Papadakis, Siegfried Rasthofer, Alexandre Bartel, Damien Octeau, Jacques Klein, and Le Traon. 2017. Static analysis of android apps: A systematic literature review. Information and Software Technology 88 (2017), 67–95.

Digital Library

[22]

Li Li, Tegawendé F Bissyandé, Haoyu Wang, and Jacques Klein. 2018. Cid: Automating the detection of api-related compatibility issues in android apps. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 153–163.

Digital Library

[23]

Li Li, Jun Gao, Tegawendé F Bissyandé, Lei Ma, Xin Xia, and Jacques Klein. 2018. Characterising deprecated android apis. In Proceedings of the 15th International Conference on Mining Software Repositories. 254–264.

Digital Library

[24]

Mario Linares-Vásquez, Gabriele Bavota, Carlos Bernal-Cárdenas, Massimiliano Di Penta, Rocco Oliveto, and Denys Poshyvanyk. 2013. Api change and fault proneness: A threat to the success of android apps. In Proceedings of the 2013 9th joint meeting on foundations of software engineering. 477–487.

Digital Library

[25]

Mario Linares-Vásquez, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Denys Poshyvanyk. 2014. How do api changes trigger stack overflow discussions? a study on the android sdk. In proceedings of the 22nd International Conference on Program Comprehension. 83–94.

Digital Library

[26]

Pei Liu, Li Li, Yichun Yan, Mattia Fazzini, and John Grundy. 2021. Identifying and characterizing silently-evolved methods in the android API. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2021). IEEE, 308–317.

Digital Library

[27]

Pei Liu, Yanjie Zhao, Haipeng Cai, Mattia Fazzini, John Grundy, and Li Li. 2022. Automatically Detecting API-induced Compatibility Issues in Android Apps: A Comparative Analysis (Replicability Study). arXiv preprint arXiv:2205.15561(2022).

[28]

Yue Liu, Chakkrit Tantithamthavorn, Li Li, and Yepang Liu. 2022. Deep Learning for Android Malware Defenses: a Systematic Literature Review. Comput. Surveys (jun 2022). https://doi.org/10.1145/3544968

Digital Library

[29]

William Martin, Federica Sarro, Yue Jia, Yuanyuan Zhang, and Mark Harman. 2016. A survey of app store analysis for software engineering. IEEE transactions on software engineering 43, 9 (2016), 817–847.

[30]

Tyler McDonnell, Baishakhi Ray, and Miryung Kim. 2013. An empirical study of api stability and adoption in the android ecosystem. In 2013 IEEE International Conference on Software Maintenance. IEEE, 70–79.

Digital Library

[31]

Patrick Mutchler, Yeganeh Safaei, Adam Doupé, and John Mitchell. 2016. Target fragmentation in Android apps. In 2016 IEEE Security and Privacy Workshops (SPW). IEEE, 204–213.

[32]

Meiyappan Nagappan and Emad Shihab. 2016. Future trends in software engineering research for mobile apps. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 5. IEEE, 21–32.

[33]

Fatih Nayebi, Jean-Marc Desharnais, and Alain Abran. 2012. The state of the art of mobile application usability evaluation. In 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE, 1–4.

[34]

Juliana Oliveira, Deise Borges, Thaisa Silva, Nelio Cacho, and Fernando Castor. 2018. Do android developers neglect error handling? a maintenance-Centric study on the relationship between android abstractions and uncaught exceptions. Journal of Systems and Software 136 (2018), 1–18.

Digital Library

[35]

Carlos Pacheco and Michael D Ernst. 2007. Randoop: feedback-directed random testing for Java. In Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion. 815–816.

[36]

Jibesh Patra and Michael Pradel. 2016. Learning to fuzz: Application-independent fuzz testing with probabilistic, generative models of input data. TU Darmstadt, Department of Computer Science, Tech. Rep. TUD-CS-2016-14664 (2016).

[37]

Xiaoxue Ren, Xinyuan Ye, Zhenchang Xing, Xin Xia, Xiwei Xu, Liming Zhu, and Jianling Sun. 2020. API-misuse detection driven by fine-grained API-constraint knowledge graph. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 461–472.

Digital Library

[38]

Jordan Samhi, Jun Gao, Nadia Daoudi, Pierre Graux, Henri Hoyez, Xiaoyu Sun, Kevin Allix, Tegawendé F Bissyandé, and Jacques Klein. 2022. JuCify: a step towards Android code unification for enhanced static analysis. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE). IEEE, 1232–1244.

Digital Library

[39]

Simone Scalabrino, Gabriele Bavota, Mario Linares-Vásquez, Michele Lanza, and Rocco Oliveto. 2019. Data-driven solutions to detect api compatibility issues in android: an empirical study. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). IEEE, 288–298.

Digital Library

[40]

Simone Scalabrino, Gabriele Bavota, Mario Linares-Vásquez, Valentina Piantadosi, Michele Lanza, and Rocco Oliveto. 2020. API compatibility issues in Android: Causes and effectiveness of data-driven detection techniques. Empirical Software Engineering 25, 6 (2020), 5006–5046.

Digital Library

[41]

Stack OverFlow. 2021. Permissions needed for NotificationManager. https://stackoverflow.com/questions/41308512/permissions-needed-for-notificationmanager. Online; accessed 27 January 2021.

[42]

Ai-Fen Sui, Wen Tang, Jian Jun Hu, and Ming Zhu Li. 2011. An effective fuzz input generation method for protocol testing. In 2011 IEEE 13th International Conference on Communication Technology. IEEE, 728–731.

[43]

Xiaoyu Sun, Xiao Chen, Kui Liu, Sheng Wen, Li Li, and John Grundy. 2021. Characterizing Sensor Leaks in Android Apps. In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, 498–509.

[44]

Xiaoyu Sun, Li Li, Tegawendé F Bissyandé, Jacques Klein, Damien Octeau, and John Grundy. 2021. Taming reflection: An essential step toward whole-program analysis of android apps. ACM Transactions on Software Engineering and Methodology (TOSEM) 30, 3(2021), 1–36.

Digital Library

[45]

Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 2010. Soot: A Java bytecode optimization framework. In CASCON First Decade High Impact Papers. 214–224.

Digital Library

[46]

Sebastian Vogl, Sebastian Schweikl, Gordon Fraser, Andrea Arcuri, Jose Campos, and Annibale Panichella. 2021. EVOSUITE at the SBST 2021 Tool Competition. In 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST). IEEE, 28–29.

[47]

Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2016. Taming Android fragmentation: Characterizing and detecting compatibility issues for Android apps. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 226–237.

Digital Library

[48]

Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2019. Pivot: learning API-device correlations to facilitate Android compatibility issue detection. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 878–888.

Digital Library

[49]

Lili Wei, Yepang Liu, Shing-Chi Cheung, Huaxun Huang, Xuan Lu, and Xuanzhe Liu. 2018. Understanding and detecting fragmentation-induced compatibility issues for android apps. IEEE Transactions on Software Engineering 46, 11 (2018), 1176–1199.

[50]

Hao Xia, Yuan Zhang, Yingtian Zhou, Xiaoting Chen, Yang Wang, Xiangyu Zhang, Shuaishuai Cui, Geng Hong, Xiaohan Zhang, Min Yang, 2020. How Android developers handle evolution-induced API compatibility issues: a large-scale study. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 886–898.

Digital Library

[51]

Mu Zhang, Yue Duan, Heng Yin, and Zhiruo Zhao. 2014. Semantics-aware android malware classification using weighted contextual api dependency graphs. In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security. 1105–1116.

Digital Library

[52]

Qian Zhang, Jiyuan Wang, Muhammad Ali Gulzar, Rohan Padhye, and Miryung Kim. 2020. Bigfuzz: Efficient fuzz testing for data analytics using framework abstraction. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 722–733.

Digital Library

[53]

Tao Zhang, Jerry Gao, Jing Cheng, and Tadahiro Uehara. 2015. Compatibility testing service for mobile applications. In 2015 IEEE Symposium on Service-Oriented System Engineering. IEEE, 179–186.

Digital Library

[54]

Yanjie Zhao, Li Li, Kui Liu, and John Grundy. 2022. Towards Automatically Repairing Compatibility Issues in Published Android Apps. In The 44th International Conference on Software Engineering (ICSE 2022).

[55]

Jing Zhou and Robert J Walker. 2016. API deprecation: a retrospective analysis and detection method for code examples on the web. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 266–277.

Digital Library

[56]

Behrouz Zolfaghari, Reza M Parizi, Gautam Srivastava, and Yoseph Hailemariam. 2021. Root causing, detecting, and fixing flaky tests: state of the art and future roadmap. Software: Practice and Experience 51, 5 (2021), 851–867.

Cited By

Yu XLiu LHu XKeung JXia XLo DChristakis MPradel M(2024)Practitioners’ Expectations on Automated Test GenerationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680386(1618-1630)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680386
Ma GPei YChen LGan CZhang HLiang HZhang T(2024)Effective Unit Test Generation for Android Apps2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00085(820-832)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00085
Zhao YZhang YWang HLin ZLiao X(2023)Potential Risks Arising from the Absence of Signature Verification in Miniapp PluginsProceedings of the 2023 ACM Workshop on Secure and Trustworthy Superapps10.1145/3605762.3624433(59-64)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3605762.3624433
Show More Cited By

Index Terms

Mining Android API Usage to Generate Unit Test Cases for Pinpointing Compatibility Issues
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools

Index terms have been assigned to the content through auto-classification.

Recommendations

Towards automatically repairing compatibility issues in published Android apps
ICSE '22: Proceedings of the 44th International Conference on Software Engineering

The heavy fragmentation of the Android ecosystem has led to severe compatibility issues with apps, including those that crash at runtime or cannot be installed on certain devices but work well on other devices. To address this problem, various approaches ...
Android API Field Evolution and Its Induced Compatibility Issues
ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement

Background: The continuous evolution of the Android operating system necessitates regular API updates, which may affect the functionality of Android apps. Recent studies investigated API evolution to ensure the reliability of Android apps; however, they ...
Detecting Android API Compatibility Issues With API Differences
Android application programming interface (API) enables app developers to harness the functionalities of Android devices by interfacing with services and hardware using a Software Development Kit (SDK). However, API frequently evolves together with its ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

October 2022

2006 pages

ISBN:9781450394758

DOI:10.1145/3551349

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 January 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ASE '22

ASE '22: 37th IEEE/ACM International Conference on Automated Software Engineering

October 10 - 14, 2022

MI, Rochester, USA

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
173
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)3

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu XLiu LHu XKeung JXia XLo DChristakis MPradel M(2024)Practitioners’ Expectations on Automated Test GenerationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680386(1618-1630)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680386
Ma GPei YChen LGan CZhang HLiang HZhang T(2024)Effective Unit Test Generation for Android Apps2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00085(820-832)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00085
Zhao YZhang YWang HLin ZLiao X(2023)Potential Risks Arising from the Absence of Signature Verification in Miniapp PluginsProceedings of the 2023 ACM Workshop on Secure and Trustworthy Superapps10.1145/3605762.3624433(59-64)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3605762.3624433
Sun XChen XLiu YGrundy JLi L(2023)Taming Android Fragmentation Through Lightweight Crowdsourced TestingIEEE Transactions on Software Engineering10.1109/TSE.2023.326632449:6(3599-3615)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3266324
Liu YChen XLiu PGrundy JChen CLi L(2023)ReuNify: A Step Towards Whole Program Analysis for React Native Android Apps2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00113(1390-1402)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00113

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents