research-article

A Visual Interaction Cue Framework from Video Game Environments for Augmented Reality

Authors:

Kody R. Dillman,

Terrance Tin Hoi Mok,

Alex MitchellAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 140, Pages 1 - 12

https://doi.org/10.1145/3173574.3173714

Published: 19 April 2018 Publication History

Abstract

Based on an analysis of 49 popular contemporary video games, we develop a descriptive framework of visual interaction cues in video games. These cues are used to inform players what can be interacted with, where to look, and where to go within the game world. These cues vary along three dimensions: the purpose of the cue, the visual design of the cue, and the circumstances under which the cue is shown. We demonstrate that this framework can also be used to describe interaction cues for augmented reality applications. Beyond this, we show how the framework can be used to generatively derive new design ideas for visual interaction cues in augmented reality experiences.

Supplementary Material

ZIP File (pn1930.compressed.pdf.zip)

Download
2.53 MB

MP4 File (pn1930.mp4)

Download
233.36 MB

References

[1]

B. Avery, C. Sandor, and B. H. Thomas. 2009. Improving Spatial Perception for Augmented Reality X-Ray Vision. In 2009 IEEE Virtual Reality Conference, 79--82.

[2]

Benjamin Avery, Bruce H. Thomas, and Wayne Piekarski. 2008. User Evaluation of See-through Vision for Mobile Outdoor Augmented Reality. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '08), 69--72.

Digital Library

[3]

R. Bane and T. Hollerer. 2004. Interactive tools for virtual x-ray vision in mobile augmented reality. In Third IEEE and ACM International Symposium on Mixed and Augmented Reality, 231--239.

Digital Library

[4]

Shaowen Bardzell. 2008. Systems of Signs and Affordances: Interaction Cues in 3D Games. EnLeino, O: 191--209.

[5]

Mark Billinghurst, Adrian Clark, Gun Lee, and others. 201 A survey of augmented reality. Foundations and Trends® in Human-Computer Interaction 8, 2--3: 73--272.

Digital Library

[6]

Detroit Institute of Arts. Lumin. Lumin. Retrieved September 18, 2017 from https://www.dia.org/about/press/media-kits/lumin

[7]

Georgia Tech Research Corporation. argon.js. Retrieved September 18, 2017 from https://www.argonjs.io/

[8]

James J. Gibson. 1977. The Theory of Affordances. In Perceiving, acting, and knowing: Toward an ecological psychology, Robert Shaw and John Bransford (eds.). Lawrence Erlbaum Associates, Hillsdale, NJ, USA, 67--82.

[9]

Raphael Grasset, Alessandro Mulloni, Mark Billinghurst, and Dieter Schmalstieg. 2011. Navigation Techniques in Augmented and Mixed Reality: Crossing the Virtuality Continuum. In Handbook of Augmented Reality. Springer, New York, NY, 379--407.

[10]

Ioanna Iacovides, Anna Cox, Richard Kennedy, Paul Cairns, and Charlene Jennett. 2015. Removing the HUD: The Impact of Non-Diegetic Game Elements and Expertise on Player Involvement. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '15), 13--22.

Digital Library

[11]

1Kristine Jørgensen. 2010. Time for new terminology. Diegetic and nondiegetic sounds in computer games revisited. Teoksessa Grimshaw, Mark (toim.): Game Sound Technology and Player Interaction: Concepts and Developments. Hershey, PA: Information Science Reference: 78--97.

[12]

Kristine Jørgensen. 20 Between the Game System and the Fictional World: A Study of Computer Game Interfaces. Games and Culture 7, 2: 142--163.

[13]

Mark A. Livingston, J. Edward Swan II, Joseph L. Gabbard, Tobias H. Höllerer, Deborah Hix, Simon J. Julier, Yohan Baillot, and Dennis Brown. 2003. Resolving Multiple Occluded Layers in Augmented Reality. In Proceedings of the 2Nd IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '03), 56--. Retrieved September 17, 2017 from http://dl.acm.org/citation.cfm?id=946248.946796

Digital Library

[14]

Lowe's. In-Store Navigation. Lowe's Innovation Labs. Retrieved September 18, 2017 from http://www.lowesinnovationlabs.com/instorenavigation/

[15]

Donald A. Norman. 1988. The psychology of everyday things.(The design of everyday things). Basic Books.

[16]

M. Peacocke, R. J. Teather, J. Carette, and I. S. MacKenzie. 2015. Evaluating the effectiveness of HUDs and diegetic ammo displays in first-person shooter games. In 2015 IEEE Games Entertainment Media Conference (GEM), 1--8.

[17]

Gerhard Reitmayr and Dieter Schmalstieg. 2004. Scalable techniques for collaborative outdoor augmented reality. In 3rd IEEE and ACM international symposium on mixed and augmented reality (ISMAR'04), Arlington. Retrieved September 18, 2017 from https://www.ims.tuwien.ac.at/publications/tr-1882-02e.pdf

[18]

P. Salomoni, C. Prandi, M. Roccetti, L. Casanova, and L. Marchetti. 2016. Assessing the efficacy of a diegetic game interface with Oculus Rift. In 2016 13th IEEE Annual Consumer Communications Networking Conference (CCNC), 387--392.

[19]

C. Sandor, A. Dey, A. Cunningham, S. Barbier, U. Eck, D. Urquhart, M. R. Marner, G. Jarvis, and S. Rhee. 2010. Egocentric space-distorting visualizations for rapid environment exploration in mobile mixed reality. In 2010 IEEE Virtual Reality Conference (VR), 47--50.

Digital Library

[20]

Jesse Schell. 2014. The Art of Game Design: A Book of Lenses, Second Edition. CRC Press.

[21]

LUDOGRAPHY

[22]

2K Boston and 2K Australia. (2007). Bioshock. Game. {Windows}. (21 August 2007). 2K Games. Novato, CA, USA

[23]

Bethesda Game Studios. (2011). The Elder Scrolls V: Skyrim. Game. (11 November 2011). Bethesda Softworks. Rockville, MD, USA

[24]

BioWare Edmonton. (2014). Dragon Age: Inquisition. Game. {PlayStation 4}. (18 November 2014). Electronic Arts. Redwood City, CA, USA

[25]

Blizzard Entertainment. (2004). World of Warcraft. Game. {Windows}. (23 November 2004). Blizzard Entertainment. Irvine, CA, USA

[26]

Blizzard Entertainment. (2016). Overwatch. Game. {Windows}. (24 May 2016). Blizzard Entertainment. Irvine, CA, USA

[27]

CD Projekt. (2015). The Witcher 3: Wild Hunt. Game. {Windows}. (19 May 2015). CD Projekt RED. Warsaw, Poland

[28]

Eidos Montreal. (2014) Thief. Game. {Windows}. (February 25, 2014). Square Enix. Tokyo Japan

[29]

Guerilla Games. (2017). Horizon Zero Dawn. Game. {Playstation 4}. (February 28, 2017). Sony Interactive Entertainment. Tokyo, Japan

[30]

Halfbrick Studios. (2011). Jetpack Joyride. Game. {iOS}. (September 1, 2011). Halfbrick Studios. Brisbane, Australia

[31]

id Software. (1993). Doom. Game. {MS-DOS}. (10 December 1993). GT Interactive. New York City, NY, USA

[32]

1id Software. (2016). Doom. Game. {Windows}. (13 May 2016). Bethesda Softworks.Rockville, MD, USA

[33]

SUPERHOT Team. (2016). Superhot. Game. {Windows}. (25 February 2016). SUPERHOT Team. Łódž, Poland

[34]

Team Ico. (2005), Shadow of the Colossus. Game. {PlayStation 2}. (18 October 2005). Sony Computer Entertainment. Tokyo, Japan

[35]

Terrible Toybox. (2017). Thimbleweed Park. Game. {Windows}. (30 March 2017). Terrible Toybox. Seattle, WA, USA

[36]

Ubisoft Annecy. (2016). Steep. Game. {Windows}. (2 December 2016). Ubisoft. Rennes, France

[37]

Ubisoft Quebec. (2015. Assassin's Creed Syndicate. Game. {Windows}. (19 November 2015). Ubisoft. Rennes, France

[38]

Valve Corporation. (2009). Left 4 Dead 2. Game. {Windows}. (November 17, 2009). Valve Corporation. Bellevue, WA, US

[39]

Visceral Games. (2013). Dead Space 3. Game. {Windows}. (5 February 2013). Electronic Arts. Redwood City, CA, USA

Cited By

Karaosmanoglu SWegner MCmentowski SSteinicke F(2024)Move, React, Repeat! The Role of Continuous Cues in Immersive ExergamesProceedings of the ACM on Human-Computer Interaction10.1145/36770918:CHI PLAY(1-23)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677091
Zhan YNakajima TSakai TIshita EOhshima HRadboud University, Netherlands FMao JJose J(2024)Effect of Presentation Methods on User Experiences and Perception in VR Shopping Recommender SystemsProceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3673791.3698430(241-248)Online publication date: 8-Dec-2024
https://dl.acm.org/doi/10.1145/3673791.3698430
Gamage BHolloway LMcDowell NDo TPrice NLowery AMarriott K(2024)Vision-Based Assistive Technologies for People with Cerebral Visual Impairment: A Review and Focus StudyProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675637(1-20)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675637
Show More Cited By

Index Terms

A Visual Interaction Cue Framework from Video Game Environments for Augmented Reality
1. Human-centered computing

Recommendations

Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Developing an augmented reality racing game
INTETAIN '08: Proceedings of the 2nd international conference on INtelligent TEchnologies for interactive enterTAINment

Augmented reality (AR) makes it possible to create games in which virtual objects are overlaid on the real world, and real objects are tracked and used to control virtual ones. We describe the development of an AR racing game created by modifying an ...
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSERC

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
1,875
Total Downloads

Downloads (Last 12 months)248
Downloads (Last 6 weeks)30

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Karaosmanoglu SWegner MCmentowski SSteinicke F(2024)Move, React, Repeat! The Role of Continuous Cues in Immersive ExergamesProceedings of the ACM on Human-Computer Interaction10.1145/36770918:CHI PLAY(1-23)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677091
Zhan YNakajima TSakai TIshita EOhshima HRadboud University, Netherlands FMao JJose J(2024)Effect of Presentation Methods on User Experiences and Perception in VR Shopping Recommender SystemsProceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3673791.3698430(241-248)Online publication date: 8-Dec-2024
https://dl.acm.org/doi/10.1145/3673791.3698430
Gamage BHolloway LMcDowell NDo TPrice NLowery AMarriott K(2024)Vision-Based Assistive Technologies for People with Cerebral Visual Impairment: A Review and Focus StudyProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675637(1-20)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675637
Chen BYan XHu XKao DLiang H(2024)Impact of Tutorial Modes with Different Time Flow Rates in Virtual Reality GamesProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36512967:1(1-19)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3651296
Yu DCibulskis MMortensen EChristensen MBergström J(2024)Metrics of Motor Learning for Analyzing Movement Mapping in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642354(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642354
Sutton JLanglotz TPlopski AHornbæk K(2024)Flicker Augmentations: Rapid Brightness Modulation for Real-World Visual Guidance using Augmented RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642085(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642085
Syiem BWebber SKelly RZhou QGoncalves JVelloso E(2024)Augmented Reality at Zoo Exhibits: A Design Framework for Enhancing the Zoo ExperienceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642015(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642015
Doerr NLee BBaricova KSchmalstieg DSedlmair M(2024)Visual Highlighting for Situated Brushing and LinkingComputer Graphics Forum10.1111/cgf.1510543:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15105
Yao LBucchieri FMcArthur VBezerianos AIsenberg P(2024)User Experience of Visualizations in Motion: A Case Study and Design ConsiderationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345631931:1(174-184)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456319
Kelley B(2024)Do You See It? Aspects of Effective Visual Cueing2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00198(658-659)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00198
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents