research-article

A co-evolutionary genetic algorithms approach to detect video game bugs

Authors:

Aghyad Albaghajati,

Moataz AhmedAuthors Info & Claims

Volume 188, Issue C

https://doi.org/10.1016/j.jss.2022.111261

Published: 01 June 2022 Publication History

Abstract

Video games systems are known for their complexity, concurrency and non-determinism, which makes them prone to challenging tacit bugs. Video games development is costly and the corresponding verification process is tiresome. Testing the nondeterministic and concurrent behaviors of video games systems is not only crucial but also challenging, especially when the game state space is huge. Accordingly, typical software testing approaches are neither suitable nor effective to find related bugs. Novel automated approaches to support video game testing are needed. This problem has caught researchers’ attention recently. Approaches found in the literature have tried to address two sub problems: modeling and uncovering bugs. Colored Petri nets is known to support modeling and verifying concurrent and nondeterministic systems. Search approaches have been used in the literature to check the availability of faulty states through exploring state spaces. However, these approaches tend to lack adaptability to test different video games systems due to the limitations of the defined fitness functions, in addition to difficulties in searching huge state spaces due to exhaustive and unguided search. The availability of automated approaches that guide and direct the process of bugs finding is mandatory. Thus, in this study we address this problem as we present a solution for automated software testing using collaborative work of two genetic algorithms (i.e. co-evolutionary) agents, where our approach is applied to colored Petri nets representations of the software workflow. The results of our experiments have shown the potential of the proposed approach in effectively finding bugs automatically.

Highlights

•

Establishing a rule scheme that is based on extending colored Petri nets with OCL.

•

Proposing a co-evolutionary automated testing approach of two genetic algorithms.

•

Evaluating our testing approach via experiments revealed its ability to find bugs.

References

[1]

Aarseth, E., 2012. A narrative theory of games. In: Proceedings of the International Conference on the Foundations of Digital Games. pp. 129–133.

[2]

Abdulhameed, A., Hammad, A., Mountassir, H., Tatibouet, B., 2015. An approach to verify SysML functional requirements using Promela/SPIN. In: 2015 12th International Symposium on Programming and Systems. ISPS, pp. 1–9.

[3]

Ahumada T., Bergel A., Reproducing bugs in video games using genetic algorithms, in: 2020 IEEE Games, Multimedia, Animation and Multiple Realities Conference (GMAX), IEEE, 2020, pp. 1–6.

[4]

Albaghajati A., Detecting game bugs using genetic algorithms and reachability analysis of colored Petri nets models, (Master Thesis) King Fahd University of Petroleum and Minerals, 2021, [Online]. Available: https://eprints.kfupm.edu.sa/id/eprint/141862.

[5]

Albaghajati A.M., Ahmed M.A.K., Video game automated testing approaches: An assessment framework, IEEE Trans. Games (2020) 1.

[6]

Ali S., Iqbal M.Z., Arcuri A., Briand L.C., Generating test data from OCL constraints with search techniques, IEEE Trans. Softw. Eng. 39 (10) (2013) 1376–1402.

Digital Library

[7]

Basili V.R., Selby R.W., Comparing the effectiveness of software testing strategies, IEEE Trans. Softw. Eng. (12) (1987) 1278–1296.

[8]

Beattie A., How the video game industry is changing, 2021, Investopedia, [Online]. Available: https://www.investopedia.com/articles/investing/053115/how-video-game-industry-changing.asp.

[9]

Bertolino, A., 2007. Software Testing Research: Achievements, Challenges, Dreams. In: Future of Software Engineering. FOSE ’07, pp. 85–103.

[10]

Blow J., Game development: Harder than you think, Queue 1 (10) (2004) 28–37.

[11]

Bouali M., Rocheteau J., Barger P., Backward reachability analysis of colored petri nets, 2009.

[12]

Carver R.H., Lei Y., A general model for reachability testing of concurrent programs, in: International Conference on Formal Engineering Methods, Springer, 2004, pp. 76–98.

[13]

Chan B., Denzinger J., Gates D., Loose K., Buchanan J., Evolutionary behavior testing of commercial computer games, in: Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753), Vol. 1, IEEE, 2004, pp. 125–132.

[14]

Cho S.M., Hong H.S., Cha S.D., Safety analysis using coloured Petri nets, in: Proceedings 1996 Asia-Pacific Software Engineering Conference, IEEE, 1996, pp. 176–183.

[15]

Colby R., Colby R.S., Game design documentation: four perspectives from independent game studios, Commun. Des. Quart. Rev. 7 (3) (2019) 5–15.

[16]

Correa A., Werner C., Applying refactoring techniques to UML/OCL models, in: International Conference on the Unified Modeling Language, Springer, 2004, pp. 173–187.

[17]

Doungsa-ard C., Dahal K.P., Hossain M.A., Suwannasart T., An automatic test data generation from UML state diagram using genetic algorithm, IEEE, 2007.

[18]

Emanuelle, F., Menezes, R., Braga, M., 2006. Using Genetic algorithms for test plans for functional testing. In: 44th ACM SE Proceeding. pp. 140–145.

[19]

Felder M., Ghezzi C., Pezzé M., High-level timed Petri nets as a kernel for executable specifications, Real-Time Syst. 5 (2) (1993) 235–248.

[20]

Fortin F.-A., De Rainville F.-M., Gardner M.-A., Parizeau M., Gagné C., DEAP: Evolutionary algorithms made easy, J. Mach. Learn. Res. 13 (2012) 2171–2175.

Digital Library

[21]

GamesIndustry.biz F.-A., Gamesindustry.biz presents... The year in numbers 2020, 2021, https://www.gamesindustry.biz/articles/2020-12-21-gamesindustry-biz-presents-the-year-in-numbers-2020. (Accessed April 2021).

[22]

Ganty P., Majumdar R., Algorithmic verification of asynchronous programs, ACM Trans. Program. Lang. Syst. (TOPLAS) 34 (1) (2012) 6.

[23]

Gogolla M., Büttner F., Richters M., USE: A UML-based specification environment for validating UML and OCL, Sci. Comput. Program. 69 (1–3) (2007) 27–34.

[24]

Gupta N.K., Rohil M.K., Using genetic algorithm for unit testing of object oriented software, in: 2008 First International Conference on Emerging Trends in Engineering and Technology, IEEE, 2008, pp. 308–313.

[25]

Holloway L.E., Krogh B.H., Giua A., A survey of Petri net methods for controlled discrete event systems, Discrete Event Dyn. Syst. 7 (2) (1997) 151–190.

Digital Library

[26]

Hwang G.-H., Tai K.-C., Huang T.-L., Reachability testing: An approach to testing concurrent software, in: Proceedings of 1st Asia-Pacific Software Engineering Conference, IEEE, 1994, pp. 246–255.

[27]

Jensen K., Kristensen L.M., Coloured Petri Nets: Modelling and Validation of Concurrent Systems, Springer Science & Business Media, 2009.

[28]

Jensen K., Kristensen L.M., Wells L., Coloured Petri nets and CPN tools for modelling and validation of concurrent systems, Int. J. Softw. Tools Technol. Transf. 9 (3–4) (2007) 213–254.

Digital Library

[29]

Kanode C.M., Haddad H.M., Software engineering challenges in game development, in: 2009 Sixth International Conference on Information Technology: New Generations, IEEE, 2009, pp. 260–265.

[30]

Karaçali B., Tai K.-C., Model checking based on simultaneous reachability analysis, in: International SPIN Workshop on Model Checking of Software, Springer, 2000, pp. 34–53.

[31]

Khalifa, A., Isaksen, A., Togelius, J., Nealen, A., 2016. Modifying MCTS for Human-Like General Video Game Playing. In: IJCAI. pp. 2514–2520.

[32]

Klaib M.F., A parallel tree based strategy for 3-way interaction testing, Procedia Comput. Sci. 65 (2015) 377–384.

[33]

Kostin A.E., Reachability analysis in T-invariant-less Petri nets, IEEE Trans. Autom. Control 48 (6) (2003) 1019–1024.

[34]

Kostin A.E., A reachability algorithm for general Petri nets based on transition invariants, in: International Symposium on Mathematical Foundations of Computer Science, Springer, 2006, pp. 608–621.

[35]

Kramer O., Genetic Algorithm Essentials, vol. 679, Springer, 2017.

[36]

Kuhn D.R., Wallace D.R., Gallo A.M., Software fault interactions and implications for software testing, IEEE Trans. Softw. Eng. 30 (6) (2004) 418–421.

Digital Library

[37]

Last M., Eyal S., Kandel A., Effective black-box testing with genetic algorithms, in: Haifa Verification Conference, Springer, 2005, pp. 134–148.

[38]

Mayr E.W., An algorithm for the general Petri net reachability problem, SIAM J. Comput. 13 (3) (1984) 441–460.

[39]

McDonald C., Schmalz M., Monheim A., Keating S., Lewin K., Cifaldi F., Lee J.H., Describing, organizing, and maintaining video game development artifacts, J. Assoc. Inf. Sci. Technol. (2020).

[40]

Mozgovoy M., Pyshkin E., A comprehensive approach to quality assurance in a mobile game project, in: Proceedings of the 14th Central and Eastern European Software Engineering Conference Russia, in: CEE-SECR ’18, Association for Computing Machinery, New York, NY, USA, 2018.

[41]

Murata T., Petri nets: Properties, analysis and applications, Proc. IEEE 77 (4) (1989) 541–580.

[42]

Murphy-Hill, E., Zimmermann, T., Nagappan, N., 2014. Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development?. In: Proceedings of the 36th International Conference on Software Engineering. pp. 1–11.

[43]

Offutt J., Thummala S., Testing concurrent user behavior of synchronous web applications with Petri nets, Softw. Syst. Model. 18 (2) (2019) 913–936.

[44]

Ortega J., Shaker N., Togelius J., Yannakakis G.N., Imitating human playing styles in super mario bros, Entertain. Comput. 4 (2) (2013) 93–104.

[45]

Orvosh D., Davis L., Using a genetic algorithm to optimize problems with feasibility constraints, in: Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence, IEEE, 1994, pp. 548–553.

[46]

Ostrowski M., Aroudj S., Automated regression testing within video game development, GSTF J. Comput. (JoC) 3 (2) (2013) 1–5.

[47]

Pommereau F., ZINC: a compiler for” any language”-coloured Petri nets, 2018.

[48]

Reuter, C., Göbel, S., Steinmetz, R., 2015. Detecting structural errors in scene-based Multiplayer Games using automatically generated Petri Nets. In: Foundations of Digital Games, Pacific Grove, USA.

[49]

Rodríguez A., Rutle A., Kristensen L.M., Durán F., A foundation for the composition of multilevel domain-specific languages, in: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), IEEE, 2019, pp. 88–97.

[50]

Rozier K.Y., Linear temporal logic symbolic model checking, Comp. Sci. Rev. 5 (2) (2011) 163–203.

[51]

Salcedo-Sanz S., A survey of repair methods used as constraint handling techniques in evolutionary algorithms, Comp. Sci. Rev. 3 (3) (2009) 175–192.

[52]

Salen K., Tekinbaş K.S., Zimmerman E., Rules of Play: Game Design Fundamentals, MIT Press, 2004.

Digital Library

[53]

Salge, C., Lipski, C., Mahlmann, T., Mathiak, B., 2008. Using genetically optimized artificial intelligence to improve gameplaying fun for strategical games. In: Proceedings of the 2008 ACM SIGGRAPH Symposium on Video Games. pp. 7–14.

[54]

Schultz C.P., Bryant R.D., Game Testing: All in One, Stylus Publishing, LLC, 2016.

[55]

Sicart M., Defining game mechanics, Game Stud. 8 (2) (2008) n.

[56]

Soeken M., Wille R., Kuhlmann M., Gogolla M., Drechsler R., Verifying UML/OCL models using boolean satisfiability, in: 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE 2010), IEEE, 2010, pp. 1341–1344.

[57]

Spanoudakis G., Kasis K., Significance of inconsistencies in UML models, Interaction 1 (2000) 1.

[58]

Syswerda G., Simulated crossover in genetic algorithms, in: Foundations of Genetic Algorithms, vol. 2, Elsevier, 1993, pp. 239–255.

[59]

Takahashi K., Ono I., Satoh H., Kobayashi O., An efficient genetic algorithm for reachability problems, in: Proceedings of the Thirtieth Hawaii International Conference on System Sciences, Vol. 5, IEEE, 1997, pp. 89–98.

[60]

Taylor R.N., General-purpose algorithm for analyzing concurrent programs, Commun. ACM;(United States) 5 (1983).

[61]

Thong W.J., Ameedeen M., A survey of Petri net tools, in: Advanced Computer and Communication Engineering Technology, Springer, 2015, pp. 537–551.

[62]

Togelius J., Schmidhuber J., An experiment in automatic game design, in: 2008 IEEE Symposium on Computational Intelligence and Games, IEEE, 2008, pp. 111–118.

[63]

Varvaressos, S., Lavoie, K., Massé, A.B., Gaboury, S., Hallé, S., 2014. Automated Bug Finding in Video Games: A Case Study for Runtime Monitoring. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation. pp. 143–152.

[64]

Yessad A., Mounier I., Labat J.-M., Kordon F., Carron T., Have you found the error? A formal framework for learning game verification, in: European Conference on Technology Enhanced Learning, Springer, 2014, pp. 476–481.

[65]

Zheng Y., Xie X., Su T., Ma L., Hao J., Meng Z., Liu Y., Shen R., Chen Y., Fan C., Wuji: Automatic online combat game testing using evolutionary deep reinforcement learning, in: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), IEEE, 2019, pp. 772–784.

[66]

Ziftci, C., Cavalcanti, D., 2020. De-Flake Your Tests : Automatically Locating Root Causes of Flaky Tests in Code At Google. In: 2020 IEEE International Conference on Software Maintenance and Evolution. ICSME, pp. 736–745.

Cited By

Roca IPastor ÓCetina CArcega L(2024)Co-evolving scenarios and simulated players to locate bugs that arise from the interaction of software models of video gamesInformation and Software Technology10.1016/j.infsof.2024.107412169:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.infsof.2024.107412
Vanègue ABourcier VPetrillo FCostiou SGonzalez Boix EScholliers C(2023)Debugging Video Games: A Systematic MappingProceedings of the 1st ACM International Workshop on Future Debugging Techniques10.1145/3605155.3605865(23-30)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1145/3605155.3605865
Agrahari VShanbhag SChimalakonda SRao A(2023)A catalogue of game-specific anti-patterns based on GitHub and Game Development Stack ExchangeJournal of Systems and Software10.1016/j.jss.2023.111789204:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111789
Show More Cited By

Index Terms

A co-evolutionary genetic algorithms approach to detect video game bugs

Index terms have been assigned to the content through auto-classification.

Recommendations

Multi-objective genetic algorithms in the study of the genetic codes adaptability

Using a robustness measure based on values of the polar requirement of amino acids, Freeland and Hurst (1998) showed that less than one in one million random hypothetical codes are better than the standard genetic code. In this paper, instead of ...
An empirical study of automatically-generated tests from the perspective of test smells
SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software Engineering

Developing test code can be as or more expensive than developing production code. Commonly, developers use automated unit test generators to speed up software testing. The purpose of such tools is to shorten production time without decreasing code ...
Optimization of software testing using genetic algorithms
MACMESE'09: Proceedings of the 11th WSEAS international conference on Mathematical and computational methods in science and engineering

This paper presents the study of optimization of software testing techniques by using Genetic Algorithms (GAs) and a sufficient testing convergence condition of GAs is presented. Some new categories of genetic codes are applied in some problem ...

Comments

Information & Contributors

Information

Published In

cover image Journal of Systems and Software

Journal of Systems and Software Volume 188, Issue C

Jun 2022

289 pages

ISSN:0164-1212

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 June 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Roca IPastor ÓCetina CArcega L(2024)Co-evolving scenarios and simulated players to locate bugs that arise from the interaction of software models of video gamesInformation and Software Technology10.1016/j.infsof.2024.107412169:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.infsof.2024.107412
Vanègue ABourcier VPetrillo FCostiou SGonzalez Boix EScholliers C(2023)Debugging Video Games: A Systematic MappingProceedings of the 1st ACM International Workshop on Future Debugging Techniques10.1145/3605155.3605865(23-30)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1145/3605155.3605865
Agrahari VShanbhag SChimalakonda SRao A(2023)A catalogue of game-specific anti-patterns based on GitHub and Game Development Stack ExchangeJournal of Systems and Software10.1016/j.jss.2023.111789204:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111789
Bertolino ADe Angelis GLeotta MRicca F(2023)Introduction to the special issue on test automationJournal of Systems and Software10.1016/j.jss.2023.111751203:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111751

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents