skip to main content
10.1145/3311350.3347184acmconferencesArticle/Chapter ViewAbstractPublication Pageschi-playConference Proceedingsconference-collections
research-article

Incremental Game Mechanics Applied to Text Annotation

Published: 17 October 2019 Publication History

Abstract

We argue that the mechanics of 'Ville type Free-To-Play (F2P) games in general, and incremental games in particular, is especially suited for Games-With-A-Purpose. We demonstrate this through WordClicker, an incremental game whose mechanics is designed for text labelling. We believe the design and mechanics used are highly transferable to other games featuring annotation where game design is a challenge, such as serious games and language resourcing GWAPs. The game was tested with audiences from three popular indie gaming portals, achieving promising results both for entertainment value and learning.

Supplementary Material

MP4 File (p545-madge.mp4)

References

[1]
2014. Glossary of Metrics. (2014). https://unity3d.com/ learn/tutorials/topics/analytics/glossary-metrics
[2]
Clark Aldrich. 2005. Learning by doing: A comprehensive guide to simulations, computer games, and pedagogy in e-learning and other educational experiences. John Wiley & Sons.
[3]
Kati Alha, Elina Koskinen, Janne Paavilainen, and Juho Hamari. 2014. Free-to-Play Games: Professionals' Perspectives. (2014), 14.
[4]
Sultan A. Alharthi, Olaa Alsaedi, Zachary O. Toups, Joshua Tanenbaum, and Jessica Hammer. 2018. Playing to Wait: A Taxonomy of Idle Games. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. ACM Press, Montreal QC, Canada, 1--15.
[5]
American Psychiatric Association and others. 2013. Diagnostic and statistical manual of mental disorders (DSM-5®). American Psychiatric Pub.
[6]
Ian Bogost. 2010. Cow Clicker: The Making of Obsession (2010). http://bogost.com/writing/blog/cow_clicker_1/. (2010).
[7]
Ian Bogost. 2011. Persuasive Games: Exploitationware. (2011). https://www.gamasutra.com/view/feature/134735/ persuasive_games_exploitationware.php
[8]
Danah M Boyd and Nicole B. Ellison. 2007. Social Network Sites: Definition, History, and Scholarship. Journal of Computer-Mediated Communication 13, 1 (2007), 210--230.
[9]
Jon Chamberlain. 2014. Harnessing the intelligence of the crowd for problem solving and knowledge discovery. In AAAI conference on Human Computation and Crowdsourcing.
[10]
Jon Chamberlain, Richard Bartle, Udo Kruschwitz, Chris Madge, and Massimo Poesio. 2017. Metrics of games-with-a-purpose for NLP applications. (2017), 2.
[11]
Sande Chen. 2009. The Social Network Game Boom. (2009). http://www.gamasutra.com/view/feature/132400/ the_social_network_game_boom.php
[12]
Daniel Cook. 2008. The Princess Rescuing Application: Slides. (2008). http://www.lostgarden.com/2008/10/ princess-rescuing-application-slides.html
[13]
Seth Cooper, Firsas Khatib, Adrien Treuille, Janos Barbero, Jeehyung Lee, Michael Beenen, Andrew Leaver-Fay, David Baker, Zoran Popovic, and the Foldit Players. 2010. Predicting protein structures with a multiplayer online game. Nature 466 (2010), 756--760.
[14]
Greg Costikyan. 2014. Ethical Free-to-Play Game Design (And Why it Matters). (2014). https://www.gamasutra.com/view/feature/207779/ ethical_freetoplay_game_design_.php
[15]
Mihaly Csikszentmihalyi. 1990. Flow: The psychology of optimal performance. New York: Harper and Row.
[16]
M. Dreier, K. Wölfling, E. Duven, S. Giralt, M.E. Beutel, and K.W. Müller. 2017. Free-to-play: About addicted Whales, at risk Dolphins and healthy Minnows. Monetarization design and Internet Gaming Disorder. Addictive Behaviors 64 (Jan. 2017), 328--333.
[17]
Dagmara Dziedzic and Wojciech Wodarczyk. 2016. Making NLP games with a purpose fun to play using Free to Play mechanics: RoboCorp case study. In Proceedings of EACL-Games4NLP Workshop. 187--197.
[18]
Karen Fort, Bruno Guillaume, and Hadrien Chastant. 2014. Creating Zombilingo, a game with a purpose for dependency syntax annotation. In Proceedings of the First International Workshop on Gamification for Information Retrieval - GamifIR '14. ACM Press, Amsterdam, The Netherlands, 2--6.
[19]
Karën Fort, Bruno Guillaume, and Nicolas Lefebvre. 2017. Who wants to play Zombie? A survey of the players on ZOMBILINGO. In Games4NLP 2017-Using Games and Gamification for Natural Language Processing. 2.
[20]
Fragsworth. 2017. Clicker Heroes 2. (2017). http://www.clickerheroes2.com/paytowin.php
[21]
Victoria A. Fromkin (Ed.). 2001. Linguistics: An Introduction to Linguistic Theory. Blackwell.
[22]
John Funk. 2009. How Much Did Modern Warfare 2 Cost to Make? | The Escapist. (2009). http://www.escapistmagazine.com/news/view/96227-HowMuch-Did-Modern-Warfare-2-Cost-to-Make
[23]
Ujwal Gadiraju, Besnik Fetahu, and Ricardo Kawase. 2015. Training Workers for Improving Performance in Crowdsourcing Microtasks. In Design for Teaching and Learning in a Networked World, Gráinne Conole, Tomas Klobu car, Christoph Rensing, Johannes Konert, and Elise Lavoué (Eds.). Vol. 9307. Springer International Publishing, Cham, 100--114.
[24]
S Gainsbury, D King, Brett Abarbanel, Paul Delfabbro, and Nerilee Hing. 2015. Convergence of gambling and gaming in digital media. Melbourne, VIC: Victorian Responsible Gambling Foundation (2015).
[25]
Doublespeak Games. 2013. A Dark Room. http://adarkroom.doublespeakgames.com/. (2013).
[26]
Kris Graft. 2013. Wargaming kicks 'pay-to-win' monetization to the curb. (2013). /view/news/193520/ Wargaming_kicks_paytowin_monetization_to_the_curb.php
[27]
Doug Gross. 2010. The Facebook games that millions love (and hate) - CNN.com. (2010). http: //www.cnn.com/2010/TECH/02/23/facebook.games/index.html
[28]
Ronald Grover and Malathi Nayak. 2014. Activision plans $500 million date with 'Destiny'. Reuters (May 2014). https://www.reuters.com/article/us-activisiondestiny-idUSBREA4501F20140506
[29]
Juho Hamari, David J Shernoff, Elizabeth Rowe, Brianno Coller, Jodi Asbell-Clarke, and Teon Edwards. 2016. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior 54 (2016), 170--179.
[30]
Aljean Harmetz. 1984. Sigh of Relief on Video Games. The New York Times (Jan. 1984). https://www.nytimes.com/1984/01/10/business/sigh-ofrelief-on-video-games.html
[31]
Catherine Regina Heil, Jason S. Wu, Joey J. Lee, and Torben Schmidt. 2016. A Review of Mobile Language Learning Applications: Trends, Challenges, and Opportunities. The EuroCALL Review 24, 2 (Sept. 2016),
[32]
[32] David Helgason. 2008. Thoughts On Browser Plugin Penetration -- Unity Blog. (2008). https://blogs.unity3d.com/2008/03/31/thoughts-onbrowser-plugin-penetration/
[33]
Tom Hoggins. 2018. Video game loot boxes to be investigated by US after being blamed for rise in young gamblers. The Telegraph (Nov. 2018). https://www.telegraph.co.uk/gaming/news/loot-boxesinvestigated-us-blamed-rise-young-gamblers/
[34]
IVC. 2019. Wait Wait... Don't Play Me: The Clicker Game Genre and Configuring Everyday Temporalities -- InVisible Culture. (2019). http://ivc.lib.rochester.edu/ wait-wait-dont-play-me-the-clicker-game-genre-andconfiguring-everyday-temporalities/
[35]
David Jurgens and Roberto Navigli. 2014. It's all fun and games until someone annotates: Video games with a purpose for linguistic annotation. TACL 2 (2014), 449--464.
[36]
Clara Herlina Karjo and Wiwik Andreani. 2018. Learning Foreign Languages With Duolingo and Memrise. In Proceedings of the 2018 International Conference on Distance Education and Learning ICDEL '18. ACM Press, Beijing, China, 109--112.
[37]
Imran Khaliq and Blair Purkiss. 2015. A study of interaction in idle games & perceptions on the definition of a game. In Games Entertainment Media Conference (GEM), 2015 IEEE. IEEE, 1--6.
[38]
Roisin Kiberd. 2016. Cookie Clicker, the Internet's most pointlessly addictive game, is also its most subversive. (Jan. 2016). https://kernelmag.dailydot.com/issue-sections/staffeditorials/15694/cookie-clicker-capitalist-dystopia/
[39]
Alexander King. 2015. Numbers Getting Bigger: What Are Incremental Games, and Why Are They Fun? (2015). https://gamedevelopment.tutsplus.com/articles/ numbers-getting-bigger-what-are-incremental-gamesand-why-are-they-fun--cms-23925
[40]
Ben Kirman. 2010. Emergence and playfulness in social games. In Proceedings of the 14th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek '10. ACM Press, Tampere, Finland, 71.
[41]
Zoe Kleinman. 2009. Free games attract high prices. (Nov. 2009). http://news.bbc.co.uk/1/hi/technology/8376392.stm
[42]
Raph Koster. 2013. Theory of fun for game design. " O'Reilly Media, Inc.".
[43]
Mathieu Lafourcade. 2007. Making people play for Lexical Acquisition with the JeuxDeMots prototype. (2007), 8.
[44]
Mathieu Lafourcade, Alain Joubert, and Nathalie Le Brun. 2015. Games with a Purpose (GWAPS). John Wiley & Sons.
[45]
Chris Lewis, Noah Wardrip-Fruin, and Jim Whitehead. 2012a. Motivational game design patterns of 'ville games. In Proceedings of the International Conference on the Foundations of Digital Games - FDG '12. ACM Press, Raleigh, North Carolina, 172.
[46]
Chris Lewis, Noah Wardrip-Fruin, and Jim Whitehead. 2012b. Motivational game design patterns of 'Ville games. In Proceedings of the International Conference on the Foundations of Digital Games. ACM, 172--179.
[47]
Pascal Luban. 2011. The Design of Free-To-Play Games: Part 1. (2011). https://www.gamasutra.com/view/feature/ 134920/the_design_of_freetoplay_games_.php
[48]
Will Luton. 2013. Free-to-play: Making money from games you give away. New Riders.
[49]
Joseph Macey and Juho Hamari. 2018. Investigating relationships between video gaming, spectating esports, and gambling. Computers in Human Behavior 80 (March 2018), 344--353.
[50]
Chris Madge, Jon Chamberlain, Udo Kruschwitz, and Massimo Poesio. 2017. Experiment-Driven Development of a GWAP for Marking Segments in Text. In Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play CHI PLAY '17 Extended Abstracts. ACM Press, Amsterdam, The Netherlands, 397--404.
[51]
Chris Madge, Juntao Yu, Richard Bartle, Jon Chamberlain, Udo Kruschwitz, and Massimo Poesio. 2019. Crowdsourcing and Aggregating Nested Markable Annotations. In Proc. of ACL. Association for Computational Linguistics(ACL), Firenze, 797--807. https://www.aclweb.org/anthology/P19--1077
[52]
Eddie Makuch. 2017. Dead Space 2 Dev Says Game Cost $60M To Make And Sold Millions, But Failed Commercially. (2017). https://www.gamespot.com/articles/dead-space-2-devsays-game-cost-60m-to-make-and-so/1100--6454150/
[53]
Thomas W Malone. 1981. Toward a theory of intrinsically motivating instruction. Cognitive science 5, 4 (1981), 333--369.
[54]
Richard E. Mayer. 2002. Rote Versus Meaningful Learning. Theory Into Practice 41, 4 (Nov. 2002), 226--232.
[55]
Brice Morrison. 2011. A Necessary Evil: Grinding in Games. (2011). https://www.gamasutra.com/blogs/BriceMorrison/ 20110211/88931/A_Necessary_Evil_Grinding_in_Games.php
[56]
Janne Paavilainen, Juho Hamari, Jaakko Stenros, and Jani Kinnunen. 2013. Social Network Games: Players' Perspectives. Simulation & Gaming 44, 6 (Dec. 2013), 794--820.
[57]
Anthony Pecorella. 2016. The Math of Idle Games, Part I. (Oct. 2016). https: //blog.kongregate.com/the-math-of-idle-games-part-i/
[58]
Massimo Poesio, Jon Chamberlain, Udo Kruschwitz, Livio Robaldo, and Luca Ducceschi. 2013. Phrase detectives: Utilizing collective intelligence for internet-scale language resource creation. ACM TiiS 3, 1 (2013), 3.
[59]
Mike Rose. 2013. Chasing the Whale: Examining the ethics of free-to-play games. (2013). https://www.gamasutra.com/view/feature/195806/ chasing_the_whale_examining_the_.php
[60]
Burr Settles and Brendan Meeder. 2016. A Trainable Spaced Repetition Model for Language Learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, 1848--1858.
[61]
Dong-Hee Shin and Youn-Joo Shin. 2011. Why do people play social network games? Computers in Human Behavior 27, 2 (March 2011), 852--861.
[62]
W. Barlow Soper and Mark J. Miller. 1983. Junk-Time Junkies: An Emerging Addiction Among Students. The School Counselor 31, 1 (1983), 40--43. https://www.jstor.org/stable/23900931
[63]
Joshua Tanenbaum and Jim Bizzocchi. 2009. Rock Band: a case study in the design of embodied interface experience. In Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games - Sandbox '09. ACM Press, New Orleans, Louisiana, 127.
[64]
Jason Tanz. 2011. The Curse of Cow Clicker: How a Cheeky Satire Became a Videogame Hit. Wired 20, 1 (Dec. 2011). https://www.wired.com/2011/12/ff-cowclicker/
[65]
Paul Tassi. 2014. Why It's Scary When 0.15% Mobile Gamers Bring In 50% Of The Revenue. (2014). https://www.forbes.com/sites/insertcoin/2014/03/01/ why-its-scary-when-0--15-mobile-gamers-bring-in-50of-the-revenue/
[66]
Insights Team. 2015. How to Identify Whales In Your Game. (Sept. 2015). https://gameanalytics.com/blog/ how-to-identify-whales-in-your-game.html
[67]
Richard Thaler. 1980. Toward a positive theory of consumer choice. Journal of Economic Behavior & Organization 1, 1 (March 1980), 39--60.
[68]
Ginger Tsueng, Steven M. Nanis, Jennifer Fouquier, Benjamin M. Good, and Andrew I. Su. 2016. Citizen Science for Mining the Biomedical Literature. Citizen science : theory and practice 1, 2 (2016).
[69]
Kathleen Tuite. 2014. GWAPs: Games with a Problem. (2014), 7.
[70]
Noortje Venhuizen, Kilian Evang, Valerio Basile, and Johan Bos. 2013. Gamification for word sense labeling. In Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013).
[71]
Luis Von Ahn. 2006. Games with a purpose. Computer 39, 6 (2006), 92--94.
[72]
Luis von Ahn and Laura Dabbish. 2004. Labeling images with a computer game. In Proceedings of the 2004 conference on Human factors in computing systems - CHI '04. ACM Press, Vienna, Austria, 319--326.
[73]
Luis Von Ahn and Laura Dabbish. 2008. Designing games with a purpose. Commun. ACM 51, 8 (2008), 58--67.
[74]
Luis Von Ahn, Mihir Kedia, and Manuel Blum. 2006. Verbosity: a game for collecting common-sense facts. In Proceedings of the SIGCHI conference on Human Factors in computing systems. ACM, 75--78.
[75]
David Xicota. 2014. Free to play and its Key Performance Indicators. (2014). http: //www.gamasutra.com/blogs/DavidXicota/20140527/218550/ Free_to_play_and_its_Key_Performance_Indicators.php
[76]
Cevat Yerli. 2008. GC 2008: Crysis Cost 22 Million to Make. (2008). https://www.ign.com/articles/2008/08/19/ gc-2008-crysis-cost-22-million-to-make
[77]
José P Zagal, Staffan Björk, and Chris Lewis. 2013. Dark Patterns in the Design of Games. (2013), 8.
[78]
Amir Zeldes. 2017. The GUM corpus: creating multilayer resources in the classroom. Language Resources and Evaluation 51, 3 (Sept. 2017), 581--612.
[79]
David Zendle and Paul Cairns. 2018. Video game loot boxes are linked to problem gambling: Results of a large-scale survey. PLOS ONE 13, 11 (Nov. 2018), e0206767.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI PLAY '19: Proceedings of the Annual Symposium on Computer-Human Interaction in Play
October 2019
680 pages
ISBN:9781450366885
DOI:10.1145/3311350
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. game design
  2. games with a purpose
  3. incremental games
  4. language resourcing
  5. user training

Qualifiers

  • Research-article

Funding Sources

Conference

CHI PLAY '19
Sponsor:

Acceptance Rates

CHI PLAY '19 Paper Acceptance Rate 51 of 181 submissions, 28%;
Overall Acceptance Rate 421 of 1,386 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)44
  • Downloads (Last 6 weeks)4
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media