skip to main content
10.1145/3583131.3590351acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article
Open access

ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Design

Published: 12 July 2023 Publication History

Abstract

Large language models (LLMs) have taken the scientific world by storm, changing the landscape of natural language processing and human-computer interaction. These powerful tools can answer complex questions and, surprisingly, perform challenging creative tasks (e.g., generate code and applications to solve problems, write stories, pieces of music, etc.). In this paper, we present a collaborative game design framework that combines interactive evolution and large language models to simulate the typical human design process. We use the former to exploit users' feedback for selecting the most promising ideas and large language models for a very complex creative task---the recombination and variation of ideas. In our framework, the process starts with a brief and a set of candidate designs, either generated using a language model or proposed by the users. Next, users collaborate on the design process by providing feedback to an interactive genetic algorithm that selects, recombines, and mutates the most promising designs. We evaluated our framework on three game design tasks with human designers who collaborated remotely.

Supplementary Material

PDF File (p1383-lanzi-suppl.pdf)
Supplemental material.

References

[1]
Philip Bontrager, Wending Lin, Julian Togelius, and Sebastian Risi. 2018. Deep Interactive Evolution. In Computational Intelligence in Music, Sound, Art and Design - 7th International Conference, EvoMUSART 2018, Parma, Italy, April 4--6, 2018, Proceedings (Lecture Notes in Computer Science, Vol. 10783), Antonios Liapis, Juan Jesús Romero Cardalda, and Anikó Ekárt (Eds.). Springer, 267--282.
[2]
A.M. Brintrup, J. Ramsden, H. Takagi, and A. Tiwari. 2008. Ergonomic Chair Design by Fusing Qualitative and Quantitative Criteria Using Interactive Genetic Algorithms. Evolutionary Computation, IEEE Transactions on 12, 3 (June 2008), 343 --354.
[3]
Andrew R. Brown, Matthew Horrigan, Arne Eigenfeldt, Toby Gifford, Daniel Field, and Jon McCormack. 2018. Interacting with Musebots. In 18th International Conference on New Interfaces for Musical Expression, NIME 2018, Blacksburg, VA, USA, June 3--6, 2018. nime.org, 19--24. http://www.nime.org/proceedings/2018/nime2018_paper0004.pdf
[4]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, and al. 2020. Language Models are Few-Shot Learners.
[5]
Luigi Cardamone, Pier Luca Lanzi, and Daniele Loiacono. 2015. TrackGen: An interactive track generator for TORCS and Speed-Dreams. Appl. Soft Comput. 28 (2015), 550--558.
[6]
Luigi Cardamone, Daniele Loiacono, and Pier Luca Lanzi. 2011. Interactive evolution for the procedural generation of tracks in a high-end racing game. In 13th Annual Genetic and Evolutionary Computation Conference, GECCO 2011, Proceedings, Dublin, Ireland, July 12--16, 2011, Natalio Krasnogor and Pier Luca Lanzi (Eds.). ACM, 395--402.
[7]
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. 2020. Generative Pretraining From Pixels. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13--18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 1691--1703. http://proceedings.mlr.press/v119/chen20s.html
[8]
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondede Oliveira Pinto, and al. 2021. Evaluating Large Language Models Trained on Code.
[9]
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems.
[10]
Richard Dawkins. 1986. The Blind Watchmaker: Why the Evidence of Evolution Reveals a Universe without Design. Norton & Company, Inc.
[11]
Marc Ebner, Markus Reinhardt, and Jürgen Albert. 2005. Evolution of Vertex and Pixel Shaders. In Genetic Programming, 8th European Conference, EuroGP2005, Lausanne, Switzerland, March 30 - April 1, 2005, Proceedings (Lecture Notes in Computer Science, Vol. 3447), Maarten Keijzer, Andrea Tettamanzi, Pierre Collet, Jano I. van Hemert, and Marco Tomassini (Eds.). Springer, 261--270.
[12]
Kevin Frans. 2021. AI Charades: Language Models as Interactive Game Environments. In 2021 IEEE Conference on Games (CoG). 1--2.
[13]
Edoardo Giacomello, Pier Luca Lanzi, and Daniele Loiacono. 2019. Searching the Latent Space of a Generative Adversarial Network to Generate DOOM Levels. In IEEE Conference on Games, CoG 2019, London, United Kingdom, August 20--23, 2019. IEEE, 1--8.
[14]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139--144.
[15]
Dave Gray and Sunni Brown. 2010. Gamestorming: A Playbook for Innovators, Rulebreakers, and Changemakers. O'Reilly.
[16]
Erin J. Hastings, Ratan K. Guha, and Kenneth O. Stanley. 2009. Automatic Content Generation in the Galactic Arms Race Video Game. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (2009), 245--263.
[17]
Amy K. Hoover and Kenneth O. Stanley. 2009. Exploiting functional relationships in musical composition. Connect. Sci. 21, 2&3 (2009), 227--251.
[18]
Amy K. Hoover, Paul A. Szerlip, and Kenneth O. Stanley. 2011. Interactively evolving harmonies through functional scaffolding. In 13th Annual Genetic and Evolutionary Computation Conference, GECCO 2011, Proceedings, Dublin, Ireland, July 12--16, 2011, Natalio Krasnogor and Pier Luca Lanzi (Eds.). ACM, 387--394.
[19]
Gregory S. Hornby and Josh C. Bongard. 2012. Accelerating human-computer collaborative search through learning comparative and predictive user models. In Genetic and Evolutionary Computation Conference, GECCO '12, Philadelphia, PA, USA, July 7--11, 2012, Terence Soule and Jason H. Moore (Eds.). ACM, 225--232.
[20]
Andrew Howlett, Simon Colton, and Cameron Browne. 2010. Evolving pixel shaders for the prototype video game Subversion. In Proceedings of the 3rd International Symposium on AI and Games - A Symposium at the AISB 2010 Convention (Proceedings of the 3rd International Symposium on AI and Games - A Symposium at the AISB 2010 Convention). 41--46. AISB Symposium on AI and Games 2010 ; Conference date: 29-03-2010 Through 01-04-2010.
[21]
Shahroz Khan, Erkan Gunpinar, and Bekir Sener. 2019. GenYacht: An interactive generative design system for computer-aided yacht hull design. Ocean Engineering 191 (2019), 106462.
[22]
Hee-Su Kim and Sung-Bae Cho. 2000. Application of interactive genetic algorithm to fashion design. Engineering Applications of Artificial Intelligence 13, 6 (2000), 635 -- 644.
[23]
Hee-Su Kim and Sung-Bae Cho. 2005. Fashion Design Using Interactive Genetic Algorithm with Knowledge-based Encoding. Springer Berlin Heidelberg, Berlin, Heidelberg, 411--434.
[24]
Taras Kowaliw, Alan Dorin, and Jon McCormack. 2012. Promoting Creative Design in Interactive Evolutionary Computation. IEEE Trans. Evol. Comput. 16, 4 (2012), 523--536.
[25]
Taras Kowaliw, Jon McCormack, and Alan Dorin. 2011. An interactive electronic art system based on artificial ecosystemics. In 2011 IEEE Symposium on Artificial Life, ALIFE 2011, Paris, France, April 13--15, 2011. IEEE, 162--169.
[26]
John R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. The MIT Press.
[27]
Pier Luca Lanzi and Daniele Loiacono. 2023. ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Design. arXiv:2303.02155 [cs.AI]
[28]
Telegram FZ LLC and Telegram Messenger Inc. 2023. Telegram. https://telegram.org
[29]
Kyle Mahowald, Anna A. Ivanova, Idan A. Blank, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2023. Dissociating language and thought in large language models: a cognitive perspective. arXiv:2301.06627 [cs.CL]
[30]
Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, and Richard Evans. 2022. Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals.
[31]
P.Y. Mok, Jie Xu, X.X. Wang, J.T. Fan, Y.L. Kwok, and John H. Xin. 2013. An IGA-based design support system for realistic and practical fashion designs. Computer-Aided Design 45, 11 (2013), 1442--1458.
[32]
David Myers. 2009. In search of a minimalist game. In Proceedings of the 2009 DiGRA International Conference: Breaking New Ground: Innovation in Games, Play, Practice and Theory, DiGRA 2009, London, UK, September 1--4, 2009, Tanya Krzywinska, Helen W. Kennedy, and Barry Atkins (Eds.). Digital Games Research Association. http://www.digra.org/digital-library/publications/in-search-of-a-minimalist-game/
[33]
Andrew Nealen, Adam Saltsman, and Eddy Boxerman. 2011. Towards minimalist game design. In Foundations of Digital Games, FDG'11, Bordeaux, France, June 28 - July 1, 2011, Marc Cavazza, Katherine Isbister, and Charles Rich (Eds.). ACM, 38--45.
[34]
Don Norman. 2013. The Design Of Everyday Things. Basic Books. Revised edition.
[35]
OpenAI. 2021. ChatGPT: a Generative Pre-training Transformer Model for Conversational AI. https://openai.com/models/gpt-3-citation/.
[36]
OpenAI. 2022. DALL·E 2. https://openai.com/dall-e-2/. Retrieved 2 February 2023.
[37]
Michele Pirovano, Renato Mainetti, and Daniele Loiacono. 2015. Volcano: An interactive sword generator. In 2015 IEEE Games Entertainment Media Conference, GEM 2015, Toronto, ON, Canada, October 14--16, 2015, Elena G. Bertozzi, Bill Kapralos, Nahum D. Gershon, and Jim R. Parker (Eds.). IEEE, 1--8.
[38]
Stanislas Polu and Ilya Sutskever. 2020. Generative Language Modeling for Automated Theorem Proving.
[39]
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. (2019). https://openai.com/blog/better-language-models/
[40]
Sebastian Risi, Joel Lehman, David B. D'Ambrosio, Ryan Hall, and Kenneth O. Stanley. 2012. Combining Search-Based Procedural Content Generation and Social Gaming in the Petalz Video Game. In AIIDE, Mark Riedl and Gita Sukthankar (Eds.). The AAAI Press.
[41]
Jacob Schrum, Jake Gutierrez, Vanessa Volz, Jialin Liu, Simon Lucas, and Sebastian Risi. 2020. Interactive Evolution and Exploration within Latent Level-Design Space of Generative Adversarial Networks. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference (Cancún, Mexico) (GECCO '20). Association for Computing Machinery, New York, NY, USA, 148--156.
[42]
Jimmy Secretan and Nicholas Beato. 2008. Picbreeder: evolving pictures collabo-ratively online. In CHI. 1759--1768.
[43]
Murray Shanahan. 2022. Talking About Large Language Models. arXiv:2212.03551 [cs.CL]
[44]
Nazanin Alsadat Tabatabaei Anaraki. 2017. Fashion Design Aid System with Application of Interactive Genetic Algorithms. In Computational Intelligence in Music, Sound, Art and Design, João Correia, Vic Ciesielski, and Antonios Liapis (Eds.). Springer International Publishing, Cham, 289--303.
[45]
H. Takagi. 2001. Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation. Proc. IEEE 89, 9 (2001), 1275--1296.
[46]
Judith van Stegeren and Jakub Myundefinedliwiec. 2021. Fine-Tuning GPT-2 on Annotated RPG Quests for NPC Dialogue Generation. In Proceedings of the 16th International Conference on the Foundations of Digital Games (Montreal, QC, Canada) (FDG '21). Association for Computing Machinery, New York, NY, USA, Article 2, 8 pages.
[47]
Vanessa Volz, Jacob Schrum, Jialin Liu, Simon M. Lucas, Adam M. Smith, and Sebastian Risi. 2018. Evolving mario levels in the latent space of a deep convolutional generative adversarial network. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2018, Kyoto, Japan, July 15--19, 2018, Hernán E. Aguirre and Keiki Takadama (Eds.). ACM, 221--228.
[48]
Susanna Värtinen, Perttu Hämäläinen, and Christian Guckelsberger. 2022. Generating Role-Playing Game Quests With GPT Language Models. IEEE Transactions on Games (2022), 1--12.
[49]
Tianxiong Wang and Meiyu Zhou. 2020. A method for product form design of integrating interactive genetic algorithm with the interval hesitation time and user satisfaction. International Journal of Industrial Ergonomics 76 (2020), 102901.
[50]
Wikipedia. 2022. Design Brief (Wikipedia, The Free Encyclopedia). https://en.wikipedia.org/wiki/Design_brief
[51]
Brian G. Woolley and Kenneth O. Stanley. 2014. A Novel Human-Computer Collaboration: Combining Novelty Search with Interactive Evolution. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (Vancouver, BC, Canada) (GECCO '14). Association for Computing Machinery, New York, NY, USA, 233--240.
[52]
Bin Xu, Shangfei Wang, and Xian Li. 2010. An emotional harmony generation system. In Evolutionary Computation (CEC), 2010 IEEE Congress on. 1 --7.
[53]
Georgios N. Yannakakis and Julian Togelius. 2011. Experience-Driven Procedural Content Generation. IEEE Transactions on Affective Computing 2, 3 (2011), 147--161.
[54]
Qihao Zhu and Jianxi Luo. 2022. Generative Design Ideation: A Natural Language Generation Approach.
[55]
Peter Thorup Ølsted, Benjamin Ma, and Sebastian Risi. 2015. Interactive evolution of levels for a competitive multiplayer FPS. In 2015 IEEE Congress on Evolutionary Computation (CEC). 1527--1534.

Cited By

View all
  • (2024)Study on Cutting-Edge Technology and Metaverse Success FactorsJournal of Digital Contents Society10.9728/dcs.2024.25.6.154525:6(1545-1555)Online publication date: 30-Jun-2024
  • (2024)Position: TRUSTLLMProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692883(20166-20270)Online publication date: 21-Jul-2024
  • (2024)Enhancing Player Experience in a First-Person Shooter with Dynamic Audio Cue Adjustment Based on Gaussian Progress RegressionApplied Sciences10.3390/app14231114614:23(11146)Online publication date: 29-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '23: Proceedings of the Genetic and Evolutionary Computation Conference
July 2023
1667 pages
ISBN:9798400701191
DOI:10.1145/3583131
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. collaborative design
  2. large language models
  3. interactive evolution

Qualifiers

  • Research-article

Conference

GECCO '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,052
  • Downloads (Last 6 weeks)130
Reflects downloads up to 12 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Study on Cutting-Edge Technology and Metaverse Success FactorsJournal of Digital Contents Society10.9728/dcs.2024.25.6.154525:6(1545-1555)Online publication date: 30-Jun-2024
  • (2024)Position: TRUSTLLMProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692883(20166-20270)Online publication date: 21-Jul-2024
  • (2024)Enhancing Player Experience in a First-Person Shooter with Dynamic Audio Cue Adjustment Based on Gaussian Progress RegressionApplied Sciences10.3390/app14231114614:23(11146)Online publication date: 29-Nov-2024
  • (2024)Opportunities and Challenges in the Cultivation of Software Development Professionals in the Context of Large Language ModelsProceedings of the 2024 International Symposium on Artificial Intelligence for Education10.1145/3700297.3700342(259-267)Online publication date: 6-Sep-2024
  • (2024)Engaging and Entertaining Adolescents in Health Education Using LLM-Generated Fantasy Narrative Games and Virtual AgentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650983(1-8)Online publication date: 11-May-2024
  • (2024)C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language ModelProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642224(1-18)Online publication date: 11-May-2024
  • (2024)GPT for Games: A Scoping Review (2020-2023)2024 IEEE Conference on Games (CoG)10.1109/CoG60054.2024.10645548(1-8)Online publication date: 5-Aug-2024
  • (2024)Envisioning the incorporation of Generative Artificial Intelligence into future product design education: Insights from practitioners, educators, and studentsThe Design Journal10.1080/14606925.2024.2435703(1-21)Online publication date: 17-Dec-2024
  • (2024)Integrating AIGC with design: dependence, application, and evolution - a systematic literature reviewJournal of Engineering Design10.1080/09544828.2024.2362587(1-39)Online publication date: 6-Jun-2024
  • (2024)Exploring the application of LLM-based AI in UX design: an empirical case study of ChatGPTHuman–Computer Interaction10.1080/07370024.2024.2420991(1-33)Online publication date: 16-Nov-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media