On the use of contextual information for machine learning based test case prioritization in continuous integration development
References
[1]
Do H., Mirarab S., Tahvildari L., Rothermel G., The effects of time constraints on test case prioritization: A series of controlled experiments, IEEE Trans. Softw. Eng. 36 (5) (2010) 593–617.
[2]
Yoo S., Harman M., Regression testing minimization, selection and prioritization: A survey, Softw. Test. Verif. Reliab. 22 (2) (2012) 67–120,.
[3]
Prado Lima J.A., Vergilio S.R., Test case prioritization in continuous integration environments: A systematic mapping study, Inf. Softw. Technol. 121 (2020) 106–268,.
[4]
Spieker H., Gotlieb A., Marijan D., Mossige M., Reinforcement learning for automatic test case prioritization and selection in continuous integration, in: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, in: ISSTA 2017, ACM, New York, NY, USA, 2017, pp. 12–22,.
[5]
Prado Lima J.A., Vergilio S.R., A multi-armed bandit approach for test case prioritization in continuous integration environments, IEEE Trans. Softw. Eng. 48 (2) (2022) 453–465,.
[6]
Kuleshov V., Precup D., Algorithms for multi-armed bandit problems, J. Mach. Learn. Res. 1 (2014) 1–48.
[7]
Dietterich T.G., Machine learning for sequential data: A review, in: Caelli T., Amin A., Duin R.P.W., de Ridder D., Kamel M. (Eds.), Structural, Syntactic, and Statistical Pattern Recognition, Springer Berlin Heidelberg, Berlin, Heidelberg, 2002, pp. 15–30.
[8]
E.A. Da Roza, J.A.P. Lima, R.C. Silva, S.R. Vergilio, Machine Learning Regression Techniques for Test Case Prioritization in Continuous Integration Environment, in: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER, 2022, pp. 196–206, https://doi.org/10.1109/SANER53432.2022.00034.
[9]
X. Qu, M.B. Cohen, K.M. Woolf, Combinatorial Interaction Regression Testing: A Study of Test Case Generation and Prioritization, in: IEEE International Conference on Software Maintenance, (ISSN: 1063-6773) 2007, pp. 255–264, https://doi.org/10.1109/ICSM.2007.4362638.
[10]
S. Elbaum, A. Malishevsky, G. Rothermel, Incorporating varying test costs and fault severities into test case prioritization, in: Proceedings of the 23rd International Conference on Software Engineering, (ISSN: 0270-5257) 2001, pp. 329–338, https://doi.org/10.1109/ICSE.2001.919106.
[11]
Breiman L., Random forests, Mach. Learn. 45 (1) (2001) 5–32.
[12]
Robbins H., Some aspects of the sequential design of experiments, in: Herbert Robbins Selected Papers, Springer, 1985, pp. 169–177.
[13]
Shankar R., Sridhar D.D., A comprehensive review on test case prioritization in continuous integration platforms, Int. J. Innov. Sci. Res. Technol. 8 (4) (2023) 3223–3229.,.
[14]
Roza E.A., Prado Lima J.A., Vergilio S.R., Supplementary material - on the use of contextual information for machine learning based test case prioritization in continuous integration development, 2023, Open Science Framework, https://osf.io/btdf9/?view_only=f2d8d19d6c1b420c97a5065ba2ba9ce4.
[15]
Bajaj A., Sangwan O.P., A systematic literature review of test case prioritization using genetic algorithms, IEEE Access 7 (2019) 126355–126375,.
[16]
Walcott K.R., Soffa M.L., Kapfhammer G.M., Roos R.S., TimeAware test suite prioritization, in: Proceedings of the 2006 International Symposium on Software Testing and Analysis, ISSTA ’06, Association for Computing Machinery, New York, NY, USA, 2006, pp. 1–12.
[17]
Marijan D., Gotlieb A., Sen S., Test case prioritization for continuous regression testing: An industrial case study, in: IEEE International Conference on Software Maintenance, IEEE, 2013, pp. 540–543,.
[18]
Marijan D., Multi-perspective regression test prioritization for time-constrained environments, in: Proceedings of the 2015 IEEE International Conference on Software Quality, Reliability and Security, QRS ’15, IEEE Computer Society, Washington, DC, USA, 2015, pp. 157–162,.
[19]
Marijan D., Liaaen M., Gotlieb A., Sen S., Ieva C., TITAN: Test suite optimization for highly configurable software, in: Proceedings of the IEEE International Conference on Software Testing, Verification and Validation, in: ICST, IEEE, 2017, pp. 524–531,.
[20]
Marijan D., Gotlieb A., Liaaen M., A learning algorithm for optimizing continuous integration development and testing practice, Softw. - Pract. Exp. 49 (2) (2019) 192–213,.
[21]
Xiao L., Miao H., Zhong Y., Test case prioritization and selection technique in continuous integration development environments: a case study, Int. J. Eng. Technol. 7 (2.28) (2018) 332–336,.
[22]
Haghighatkhah A., Mäntylä M., Oivo M., Kuvaja P., Test prioritization in continuous integration environments, J. Syst. Softw. 146 (2018) 80–98,.
[23]
Busjaeger B., Xie T., Learning for test prioritization: An industrial case study, in: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, in: FSE 2016, ACM, New York, NY, USA, 2016, pp. 975–980,.
[24]
Bertolino A., Guerriero A., Miranda B., Pietrantuono R., Russo S., Learning-to-rank vs ranking-to-learn: Strategies for regression testing in continuous integration, in: 42nd International Conference on Software Engineering, ICSE ’20, ACM, New York, NY, USA, 2020, pp. 1–12,.
[25]
Bagherzadeh M., Kahani N., Briand L., Reinforcement learning for test case prioritization, IEEE Trans. Softw. Eng. 48 (8) (2022) 2836–2856,.
[26]
Prado Lima J.A., Vergilio S.R., Multi-armed bandit test case prioritization in continuous integration environments: A trade-off analysis, in: 5th Brazilian Symposium on Systematic and Automated Software Testing, SAST ’20, ACM, 2020,.
[27]
Hochreiter S., Schmidhuber J., Long short-term memory, Neural Comput. 9 (8) (1997) 1735–1780,.
[28]
Z. Bnaya, R. Puzis, R. Stern, A. Felner, Volatile Multi-Armed Bandits for Guaranteed Targeted Social Crawling, in: Proceeding of Workshops At the 27th AAAI Conference on Artificial Intelligence (Late-Breaking Developments), 2013, pp. 8–10.
[29]
Li L., Chu W., Langford J., Wang X., Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms, in: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, WSDM ’11, ACM, New York, NY, USA, 2011, pp. 297–306,.
[30]
Zhou L., A survey on contextual multi-armed bandits, 2016, arXiv:1508.03326.
[31]
Li L., Chu W., Langford J., Schapire R.E., A contextual-bandit approach to personalized news article recommendation, in: Proceedings of the 19th International Conference on World Wide Web, WWW ’10, ACM, New York, NY, USA, 2010, pp. 661–670,.
[32]
Gonçalves R., Almeida C., Lüders R., Delgado M., A new hyper-heuristic based on a contextual multi-armed bandit for many-objective optimization, in: Congress on Evolutionary Computation, in: CEC, IEEE, 2018, pp. 1–8,.
[33]
Gutowski N., Amghar T., Camp O., Chhel F., Global versus individual accuracy in contextual multi-armed bandit, in: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, SAC ’19, Association for Computing Machinery, New York, NY, USA, 2019, pp. 1647–1654,.
[34]
Geurts P., Ernst D., Wehenkel L., Extremely randomized trees, Mach. Learn. 63 (1) (2006) 3–42,.
[35]
J.-H. Kwon, I.-Y. Ko, G. Rothermel, M. Staats, Test Case Prioritization Based on Information Retrieval Concepts, in: Proceedings of the 21st Asia-Pacific Software Engineering Conference, Vol. 1, 2014, pp. 19–26, https://doi.org/10.1109/APSEC.2014.12.
[36]
H. Hemmati, F. Sharifi, Investigating NLP-Based Approaches for Predicting Manual Test Case Failure, in: Proceedings of the 11th International Conference on Software Testing, Verification and Validation, in: ICST, 2018, pp. 309–319, https://doi.org/10.1109/ICST.2018.00038.
[37]
Kruskal W.H., Wallis W.A., Use of ranks in one-criterion variance analysis, J. Amer. Statist. Assoc. 47 (260) (1952) 583–621.
[38]
Mann H.B., Whitney D.R., On a test of whether one of two random variables is stochastically larger than the other, Ann. Math. Stat. (1947) 50–60.
[39]
Colanzi T.E., Assunção W.K.G., Farah P.R., Vergilio S.R., Guizzo G., A review of ten years of the symposium on search-based software engineering, in: Nejati S., Gay G. (Eds.), Proceedings of the Search-Based Software Engineering, Springer International Publishing, Cham, 2019, pp. 42–57.
[40]
Arcuri A., Briand L., A practical guide for using statistical tests to assess randomized algorithms in software engineering, in: Proceedings of the 33rd International Conference on Software Engineering, ICSE ’11, Association for Computing Machinery, New York, NY, USA, 2011, pp. 1–10,.
[41]
Rothermel G., Untch R.H., Chu C., Harrold M.J., Test case prioritization: An empirical study, in: Proceedings of the IEEE International Conference on Software Maintenance, ICSM ’99, IEEE Computer Society, 1999, pp. 179–188,.
[42]
Tipping M.E., Bishop C.M., Probabilistic principal component analysis, J. R. Stat. Soc. Ser. B Stat. Methodol. 61 (3) (1999) 611–622,.
[43]
Marijan D., Comparative study of machine learning test case prioritization for continuous integration testing, Softw. Qual. J. (2023).
[44]
Wohlin C., Runeson P., Höst M., Ohlsson M.C., Regnell B., Wesslén A., Experimentation in Software Engineering: An Introduction, Kluwer Academic Publishers, USA, 2000,.
Recommendations
Reinforcement learning for automatic test case prioritization and selection in continuous integration
ISSTA 2017: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and AnalysisTesting in Continuous Integration (CI) involves test case prioritization, selection, and execution at each cycle. Selecting the most promising test cases to detect bugs is hard if there are uncertainties on the impact of committed code changes or, if ...
Test Case Prioritization for Continuous Regression Testing: An Industrial Case Study
ICSM '13: Proceedings of the 2013 IEEE International Conference on Software MaintenanceRegression testing in continuous integration environment is bounded by tight time constraints. To satisfy time constraints and achieve testing goals, test cases must be efficiently ordered in execution. Prioritization techniques are commonly used to ...
Comparative study of machine learning test case prioritization for continuous integration testing
AbstractThere is a growing body of research indicating the potential of machine learning to tackle complex software testing challenges. One such challenge pertains to continuous integration testing, which is highly time-constrained, and generates a large ...
Comments
Information & Contributors
Information
Published In
Elsevier B.V.
Publisher
Butterworth-Heinemann
United States
Publication History
Published: 18 July 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 27 Dec 2024