research-article

Breaking The Experience: Effects of Questionnaires in VR User Studies

Authors:

Dmitry Alexandrovsky,

Sebastian Höffner,

Jan David Smeddinck,

Rainer MalakaAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 15

https://doi.org/10.1145/3313831.3376144

Published: 23 April 2020 Publication History

Abstract

Questionnaires are among the most common research tools in virtual reality (VR) evaluations and user studies. However, transitioning from virtual worlds to the physical world to respond to VR experience questionnaires can potentially lead to systematic biases. Administering questionnaires in VR (inVRQs) is becoming more common in contemporary research. This is based on the intuitive notion that inVRQs may ease participation, reduce the Break in Presence (BIP) and avoid biases. In this paper, we perform a systematic investigation into the effects of interrupting the VR experience through questionnaires using physiological data as a continuous and objective measure of presence. In a user study (n=50), we evaluated question-asking procedures using a VR shooter with two different levels of immersion. The users rated their player experience with a questionnaire either inside or outside of VR. Our results indicate a reduced BIP for the employed inVRQ without affecting the self-reported player experience.

Supplementary Material

MP4 File (paper017vf.mp4)

Supplemental video

Download
220.17 MB

MP4 File (paper017pv.mp4)

Preview video

Download
9.68 MB

MP4 File (a17-putze-presentation.mp4)

Download
40.95 MB

References

[1]

Elinda Ai-Lim Lee, Kok Wai Wong, and Chun Che Fung. 2010. How Does Desktop Virtual Reality Enhance Learning Outcomes? A Structural Equation Modeling Approach. Computers & Education 55, 4 (Dec. 2010), 1424--1442.

Digital Library

[2]

Majed Al Zayer, Isayas B. Adhanom, Paul MacNeilage, and Eelke Folmer. 2019. The Effect of Field-of-View Restriction on Sex Bias in VR Sickness and Spatial Navigation Performance. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, 354:1--354:12.

[3]

Dmitry Alexandrovsky, Susanne Putze, Michael Bonfert, Sebastian Höffner, Pitt Michelmann, Dirk Wenig, Rainer Malaka, and Jan D. Smeddinck. 2020. Examining Design Choices of Questionnaires in VR User Studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Honolulu, HI, USA.

Digital Library

[4]

R.M. Baños, C. Botella, M. Alcañiz, V. Liaño, B. Guerrero, and B. Rey. 2004. Immersion and Emotion: Their Impact on the Sense of Presence. CyberPsychology & Behavior 7, 6 (Dec. 2004), 734--741.

[5]

Thomas Baumgartner, Dominique Speck, Denise Wettstein, Ornella Masnari, Gian Beeli, and Lutz Jäncke. 2008. Feeling Present in Arousing Virtual Reality Worlds: Prefrontal Brain Regions Differentially Orchestrate Presence Experience in Adults and Children. Frontiers in Human Neuroscience 2 (2008).

[6]

Mathias Benedek and Christian Kaernbach. 2010. A Continuous Measure of Phasic Electrodermal Activity. Journal of Neuroscience Methods 190, 1--5 (June 2010), 80--91.

[7]

Stéphane Bouchard, Stéphanie Dumoulin, Jeanne Talbot, André-Anne Ledoux, Jennifer Phillips, Johana Monthuy-Blanc, Geneviève Labonté-Chartrand, Geneviève Robillard, Matteo Cantamesse, and Patrice Renaud. 2012. Manipulating Subjective Realism and Its Impact on Presence: Preliminary Results on Feasibility and Neuroanatomical Correlates. Interacting with Computers 24, 4 (July 2012), 227--236.

Digital Library

[8]

Stéphane Bouchard, Julie St-Jacques, Geneviève Robillard, and Patrice Renaud. 2008. Anxiety Increases the Feeling of Presence in Virtual Reality. Presence: Teleoperators and Virtual Environments 17, 4 (July 2008), 376--391.

Digital Library

[9]

Doug A. Bowman and Ryan P. McMahan. 2007. Virtual Reality: How Much Immersion Is Enough? Computer 40, 7 (July 2007), 36--43.

Digital Library

[10]

Andrea Brogni, Vinoba Vinayagamoorthy, Anthony Steed, and Mel Slater. 2006. Variations in Physiological Responses of Participants During Different Stages of an Immersive Virtual Environment Experiment. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '06). ACM, NY, NY, USA, 376--382.

Digital Library

[11]

Andrea Brogni, Vinoba Vinayagamoorthy, Anthony Steed, and Mel Slater. 2007. Responses of Participants during an Immersive Virtual Environment Experience. IJVR 6, 2 (2007), 1--10.

[12]

Emily Brown and Paul Cairns. 2004. A Grounded Investigation of Game Immersion. In Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems - CHI '04. ACM Press, Vienna, Austria, 1297.

Digital Library

[13]

Karl-Erik Bystrom, Woodrow Barfield, and Claudia Hendrix. 1999. A Conceptual Model of the Sense of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 8, 2 (1999), 241--244.

Digital Library

[14]

Zekun Cao, Jason Jerald, and Regis Kopper. 2018. Visually-Induced Motion Sickness Reduction via Static and Dynamic Rest Frames. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 105--112.

[15]

Albert S. Carlin, Hunter G. Hoffman, and Suzanne Weghorst. 1997. Virtual Reality and Tactile Augmentation in the Treatment of Spider Phobia: A Case Report. Behaviour Research and Therapy 35, 2 (Feb. 1997), 153--158.

[16]

Alan Cheng, Lei Yang, and Erik Andersen. 2017. Teaching Language and Culture with a Virtual Reality Game. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 541--549.

Digital Library

[17]

Tanvir Irfan Chowdhury, Sharif Mohammad Shahnewaz Ferdous, and John Quarles. 2017. Information Recall in a Virtual Reality Disability Simulation. ACM Press, 1--10.

Digital Library

[18]

Mihaly Csikszentmihalyi. 1990. Flow: The Psychology of Optimal Experience. Harper & Row, New York.

[19]

James J. Cummings and Jeremy N. Bailenson. 2016. How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence. Media Psychology 19, 2 (April 2016), 272--309.

[20]

Cath Dillon, Edmund Keogh, and Jonathan Freeman. 2002. It's Been Emotional: Affect, Physiology, and Presence. In Proceedings of the Fifth Annual International Workshop on Presence, Porto, Portugal.

[21]

Cath Dillon, Edmund Keogh, Jonathan Freeman, and Jules Davidoff. 2000. Aroused and Immersed: The Psychophysiology of Presence. In Proceedings of 3rd International Workshop on Presence, Delft University of Technology, Delft, the Netherlands. 27--28.

[22]

Tilman Dingler, Kai Kunze, and Benjamin Outram. 2018. VR Reading UIs: Assessing Text Parameters for Reading in VR. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, NY, NY, USA, LBW094:1--LBW094:6.

Digital Library

[23]

H. Q. Dinh, N. Walker, L. F. Hodges, Chang Song, and A. Kobayashi. 1999. Evaluating the Importance of Multi-Sensory Input on Memory and the Sense of Presence in Virtual Environments. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316). 222--228.

Digital Library

[24]

Alan Dix. 2009. Human-Computer Interaction. In Encyclopedia of Database Systems. Springer, 1327--1331.

[25]

John V. Draper, David B. Kaber, and John M. Usher. 1998. Telepresence. Human Factors 40, 3 (Sept. 1998), 354--375.

[26]

Stephen R. Ellis. 1995. Nature and Origins of Virtual Environments: A Bibliographical Essay. In Readings in Human--Computer Interaction, Ronald M. Baecker, Jonathan Grudin, William A. S. Buxton, and Saul Greenberg (Eds.). Morgan Kaufmann, 913--932.

[27]

Leonard S. Feldt. 1969. A Test of the Hypothesis That Cronbach's Alpha or Kuder-Richardson Coefficent Twenty Is the Same for Two Tests. Psychometrika 34, 3 (Sept. 1969), 363--373.

[28]

Leonard S. Feldt and Seonghoon Kim. 2006. Testing the Difference Between Two Alpha Coefficients With Small Samples of Subjects and Raters. Educational and Psychological Measurement 66, 4 (Aug. 2006), 589--600.

[29]

A. S. Fernandes and S. K. Feiner. 2016. Combating VR Sickness through Subtle Dynamic Field-of-View Modification. In 2016 IEEE Symposium on 3D User Interfaces (3DUI). 201--210.

[30]

Jonathan Freeman, S. E. Avons, Ray Meddis, Don E. Pearson, and Wijnand IJsselsteijn. 2000. Using Behavioral Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion Stimuli. Presence: Teleoperators and Virtual Environments 9, 2 (April 2000), 149--164.

Digital Library

[31]

Jonathan Freeman, S. E. Avons, Don E. Pearson, and Wijnand A. IJsselsteijn. 1999. Effects of Sensory Information and Prior Experience on Direct Subjective Ratings of Presence. Presence: Teleoperators and Virtual Environments 8, 1 (Feb. 1999), 1--13.

Digital Library

[32]

Julian Frommel, Michael Weber, Katja Rogers, Julia Brich, Daniel Besserer, Leonard Bradatsch, Isabel Ortinau, Ramona Schabenberger, Valentin Riemer, and Claudia Schrader. 2015. Integrated Questionnaires: Maintaining Presence in Game Environments for Self-Reported Data Acquisition. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '15. ACM Press, London, United Kingdom, 359--368.

Digital Library

[33]

Google. 2018. Rendering Text in WebVR | Web. https://developers.google.com/web/showcase/2017/within. (Sept. 2018).

[34]

Google Developers. 2017. Designing Screen Interfaces for VR (Google I/O '17). (2017).

[35]

Nathan Navarro Griffin, James Liu, and Eelke Folmer. 2018. Evaluation of Handsbusy vs Handsfree Virtual Locomotion. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, NY, NY, USA, 211--219.

Digital Library

[36]

K. Gruchalla. 2004. Immersive Well-Path Editing: Investigating the Added Value of Immersion. In IEEE Virtual Reality 2004. 157--164.

[37]

Sandra G. Hart. 1986. NASA Task Load Index (TLX). Volume 1.0; Paper and Pencil Package. Technical Report.

[38]

Carrie Heeter. 1992. Being There: The Subjective Experience of Presence. Presence: Teleoperators and Virtual Environments 1, 2 (1992), 262--271.

[39]

Claudia Hendrix and Woodrow Barfield. 1996. Presence within Virtual Environments as a Function of Visual Display Parameters. Presence: Teleoperators and Virtual Environments 5, 3 (Jan. 1996), 274--289.

Digital Library

[40]

Claudia Mary Hendrix. 1994. Exploratory Studies on the Sense of Presence in Virtual Environments as a Function of Visual and Auditory Display Parameters. Ph.D. Dissertation.

[41]

Larry F. Hodges, Barbara O. Rothbaum, Rob Kooper, Dan Opdyke, Thomas Meyer, Johannes J. de Graaff, James S. Williford, and Max M. North. 1994. Presence as The Defining Factor in a VR Application: Virtual Reality Graded Exposure in the Treatment of Acrophobia. (1994).

[42]

Wijnand A. IJsselsteijn, Huib de Ridder, Jonathan Freeman, and Steve E. Avons. 2000. Presence: Concept, Determinants, and Measurement. In Electronic Imaging, Bernice E. Rogowitz and Thrasyvoulos N. Pappas (Eds.). San Jose, CA, 520--529.

[43]

Lutz Jäncke. 2009. Virtual Reality and the Role of the Prefrontal Cortex in Adults and Children. Frontiers in Neuroscience 3, 1 (May 2009).

[44]

Jason Jerald. 2016. The VR Book: Human-Centered Design for Virtual Reality (first edition ed.). Number 8 in ACM Books. acm, Association for Computing Machinery, New York. OCLC: 945367096.

[45]

HyeongYeop Kang, Geonsun Lee, Seongsu Kwon, Ohung Kwon, Seongpil Kim, and JungHyun Han. 2018. Flotation Simulation in a Cable-Driven Virtual Environment -- A Study with Parasailing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 632:1--632:11.

Digital Library

[46]

Pankaj Khanna, Insu Yu, Jesper Mortensen, and Mel Slater. 2006. Presence in Response to Dynamic Visual Realism: A Preliminary Report of an Experiment Study. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '06). ACM, NY, NY, USA, 364--367.

Digital Library

[47]

Tina Kjær, Christoffer B. Lillelund, Mie Moth-Poulsen, Niels C. Nilsson, Rolf Nordahl, and Stefania Serafin. 2017. Can You Cut It?: An Exploration of the Effects of Editing in Cinematic Virtual Reality. ACM Press, 1--4.

Digital Library

[48]

Jarrod Knibbe, Jonas Schjerlund, Mathias Petraeus, and Kasper Hornb\{ae}k. 2018. The Dream Is Collapsing: The Experience of Exiting VR. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 483:1--483:13.

Digital Library

[49]

E. Kristal-Boneh, M. Raifel, P. Froom, and J. Ribak. 1995. Heart Rate Variability in Health and Disease. Scandinavian Journal of Work, Environment & Health 21, 2 (1995), 85--95.

[50]

Heinz W. Krohne, Boris Egloff, Carl-Walter Kohlmann, and Anja Tausch. 1996. Untersuchungen Mit Einer Deutschen Version Der "Positive and Negative Affect Schedule"(PANAS). Diagnostica-Gottingen- 42 (1996), 139--156.

[51]

Alexander Kunst. 2019. Kinds of Video Games for VR Headsets U.S. 2017. https://www.statista.com/forecasts/790465/kinds-ofvideo-games-us-consumers-would-like-to-use-a-vrheadset-for. (2019).

[52]

Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction (2 ed.). Morgan Kaufmann, Cambridge, MA.

[53]

Jane Lessiter, Jonathan Freeman, Edmund Keogh, and Jules Davidoff. 2001. A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory. Presence: Teleoperators and Virtual Environments 10, 3 (June 2001), 282--297.

Digital Library

[54]

Benny Liebold, Michael Brill, Daniel Pietschmann, Frank Schwab, and Peter Ohler. 2017. Continuous Measurement of Breaks in Presence: Psychophysiology and Orienting Responses. Media Psychology 20, 3 (July 2017), 477--501.

[55]

Matthew Lombard and Theresa Ditton. 1997. At the Heart of It All: The Concept of Presence. Journal of Computer-Mediated Communication 3, 2 (Sept. 1997).

[56]

Matthew Lombard, Theresa B Ditton, Daliza Crane, Bill Davis, Gisela Gil-Egui, Karl Horvath, and Jessica Rossman. 2000. Measuring Presence: A Literature-Based Approach to the Development of a Standardized Paper-and-Pencil Instrument. In Third International Workshop on Presence, Delft. Delft, Netherlands, 2--4.

[57]

Katerina Mania and Alan Chalmers. 2000. A User-Centered Methodology for Investigating Presence and Task Performance. Technical Report. University of Bristol, Bristol, UK, UK.

[58]

Michael Meehan. 2001. Physiological Reaction as an Objective Measure of Presence in Virtual Environments. Ph.D. Dissertation.

Digital Library

[59]

Michael Meehan, Brent Insko, Mary Whitton, and Frederick P. Brooks, Jr. 2002. Physiological Measures of Presence in Stressful Virtual Environments. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques. ACM, NY, NY, USA, 645--652.

[60]

Marvin Minsky. 1980. Telepresence. (1980).

[61]

David Modjeska and John Waterworth. 2000. Effects of Desktop 3D World Design on User Navigation and Search Performance. In 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics. 215--220.

[62]

Michael Narayan, Leo Waugh, Xiaoyu Zhang, Pradyut Bafna, and Doug Bowman. 2005. Quantifying the Benefits of Immersion for Collaboration in Virtual Environments. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '05). ACM, NY, NY, USA, 78--81.

Digital Library

[63]

Sebastian Oberdörfer, David Heidrich, and Marc Erich Latoschik. 2019. Usability of Gamified Knowledge Learning in VR and Desktop-3D. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, 175:1--175:13.

[64]

Oculus. 2019. Introduction to Best Practices. https://developer.oculus.com/design/latest/concepts/bookbp/. (Sept. 2019).

[65]

Holger T. Regenbrecht, Thomas W. Schubert, and Frank Friedmann. 1998. Measuring the Sense of Presence and Its Relations to Fear of Heights in Virtual Environments. International Journal of Human--Computer Interaction 10, 3 (Sept. 1998), 233--249.

[66]

Richard M. Ryan, C. Scott Rigby, and Andrew Przybylski. 2006. The Motivational Pull of Video Games: A Self-Determination Theory Approach. Motivation and Emotion 30, 4 (Nov. 2006), 344--360.

[67]

Maria V. Sanchez-Vives and Mel Slater. 2005. From Presence to Consciousness through Virtual Reality. Nature Reviews Neuroscience 6, 4 (April 2005), 332.

[68]

Corina Sas and Gregory M. P. O'Hare. 2003. Presence Equation: An Investigation into Cognitive Factors Underlying Presence. Presence: Teleoperators and Virtual Environments 12, 5 (2003), 523--537.

Digital Library

[69]

Klaus R. Scherer and Heiner Ellgring. 2007. Are Facial Expressions of Emotion Produced by Categorical Affect Programs or Dynamically Driven by Appraisal? Emotion 7, 1 (Feb. 2007), 113--130.

[70]

Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoper. Virtual Environ. 10, 3 (June 2001), 266--281.

Digital Library

[71]

Peter Schulz, Dmitry Alexandrovsky, Felix Putze, Rainer Malaka, and Johannes Schöning. 2019. The Role of Physical Props in VR Climbing Environments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, 183:1--183:13.

[72]

Johannes Schumm, Marc Bächlin, Cornelia Setz, Bert Arnrich, Daniel Roggen, and Gerhard Tröster. 2008. Effect of Movements on the Electrodermal Response after a Startle Event. Methods of Information in Medicine 47, 3 (2008), 186--191.

[73]

Valentin Schwind, Pascal Knierim, Lewis Chuang, and Niels Henze. 2017. "Where's Pinky?": The Effects of a Reduced Number of Fingers in Virtual Reality. ACM Press, 507--515.

Digital Library

[74]

Valentin Schwind, Pascal Knierim, Nico Haas, and Niels Henze. 2019. Using Presence Questionnaires in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, 360:1--360:12.

[75]

Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, Nico Haas, and Niels Henze. 2017. "These Are Not My Hands!": Effect of Gender on the Perception of Avatar Hands in Virtual Reality. ACM Press, 1577--1582.

Digital Library

[76]

Valentin Schwind, Sven Mayer, Alexandre Comeau-Vermeersch, Robin Schweigert, and Niels Henze. 2018. Up to the Finger Tip: The Effect of Avatars on Mid-Air Pointing Accuracy in Virtual Reality. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '18). ACM, NY, NY, USA, 477--488.

Digital Library

[77]

Fredric Shaffer, Didier Combatalade, Erik Peper, and Zachary M. Meehan. 2016. A Guide to Cleaner Electrodermal Activity Measurements. Biofeedback 44, 2 (June 2016), 90--100.

[78]

Thomas B. Sheridan. 1992. Musings on Telepresence and Virtual Presence. Presence: Teleoperators and Virtual Environments 1, 1 (Jan. 1992), 120--126.

Digital Library

[79]

Ben Shneiderman, Catherine Plaisant, Maxine Cohen, Steven Jacobs, Niklas Elmqvist, and Nicholas Diakopoulos. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction (6 edition ed.). Pearson, Boston.

[80]

Michael J Singer, Jennifer Ehrlich, and Stephen Cinq-Mars. 1995. Task Performance in Virtual Environments: Stereoscopic versus Monoscopic Displays and Head-Coupling. Technical Report. ARMY RESEARCH INST FOR THE BEHAVIORAL AND SOCIAL SCIENCES ALEXANDRIA VA.

[81]

Richard Skarbez, Frederick P. Brooks, Jr., and Mary C. Whitton. 2017. A Survey of Presence and Related Concepts. ACM Comput. Surv. 50, 6 (Nov. 2017), 96:1--96:39.

[82]

Mel Slater. 2003. A Note on Presence Terminology. Presence connect 3, 3 (2003), 1--5.

[83]

Mel Slater. 2004. How Colorful Was Your Day? Why Questionnaires Cannot Assess Presence in Virtual Environments. Presence: Teleoperators & Virtual Environments 13, 4 (2004), 484--493.

Digital Library

[84]

Mel Slater. 2007. The Concept of Presence and Its Measurement. (July 2007).

[85]

Mel Slater, Andrea Brogni, and Anthony Steed. 2003. Physiological Responses to Breaks in Presence: A Pilot Study. In Annual International Workshop On Presence, Vol. 6. 4.

[86]

Mel Slater, Christoph Guger, Guenter Edlinger, Robert Leeb, Gert Pfurtscheller, Angus Antley, Maia Garau, Andrea Brogni, and Doron Friedman. 2006. Analysis of Physiological Responses to a Social Situation in an Immersive Virtual Environment. Presence: Teleoperators and Virtual Environments 15, 5 (Oct. 2006), 553--569.

Digital Library

[87]

Mel Slater, Pankaj Khanna, Jesper Mortensen, and Insu Yu. 2009. Visual Realism Enhances Realistic Response in an Immersive Virtual Environment. IEEE Computer Graphics and Applications 29, 3 (May 2009), 76--84.

[88]

Mel Slater, Vasilis Linakis, Martin Usoh, and Rob Kooper. 1996. Immersion, Presence and Performance in Virtual Environments: An Experiment with Tri-Dimensional Chess. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '96). ACM, NY, NY, USA, 163--172.

Digital Library

[89]

Mel Slater and Anthony Steed. 2000. A Virtual Presence Counter. Presence: Teleoperators and Virtual Environments 9, 5 (2000), 413--434.

Digital Library

[90]

Mel Slater, Martin Usoh, and Yiorgos Chrysanthou. 1995. The Influence of Dynamic Shadows on Presence in Immersive Virtual Environments. In Virtual Environments '95 (Eurographics), Martin Göbel (Ed.). Springer Vienna, 8--21.

[91]

Mel Slater, Martin Usoh, and Anthony Steed. 1994. Depth of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 3, 2 (1994), 130--144.

Digital Library

[92]

Mel Slater and Sylvia Wilbur. 1997. A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 6, 6 (1997), 603--616.

Digital Library

[93]

Michael P. Snow and Robert C. Williges. 1998. Empirical Models Based on Free-Modulus Magnitude Estimation of Perceived Presence in Virtual Environments. Human Factors 40, 3 (1998), 386--402.

[94]

Maurício Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simões Lopes. 2017. VRRRRoom: Virtual Reality for Radiologists in the Reading Room. ACM Press, 4057--4062.

Digital Library

[95]

John Towell and Elizabeth Towell. 1997. Presence in Text-Based Networked Virtual Environments or "MUDS". Presence: Teleoperators and Virtual Environments 6, 5 (Oct. 1997), 590--595.

Digital Library

[96]

Martin Usoh, Ernest Catena, Sima Arman, and Mel Slater. 2000. Using Presence Questionnaires in Reality. Presence: Teleoperators and Virtual Environments 9, 5 (2000), 497--503.

Digital Library

[97]

Marieke van Dooren, J. J. G. (Gert-Jan) de Vries, and Joris H. Janssen. 2012. Emotional Sweating across the Body: Comparing 16 Different Skin Conductance Measurement Locations. Physiology & Behavior 106, 2 (May 2012), 298--304.

[98]

Robert B. Welch. 1999. How Can We Determine If the Sense of Presence Affects Task Performance? http://dx.doi.org/10.1162/105474699566387 (1999).

Digital Library

[99]

Robert B. Welch, Theodore T. Blackmon, Andrew Liu, Barbara A. Mellers, and Lawrence W. Stark. 1996. The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence. Presence: Teleoperators and Virtual Environments 5, 3 (1996), 263--273.

Digital Library

[100]

Brenda K Wiederhold, Dong P Jang, Mayumi Kaneda, Yair Lurie, Todd May, In Y Kim, and Sun I Kim. 2003. An Investigation into Physiological Responses in Virtual Environments: An Objective Measurement of Presence. In Towards Cyberpsychology: Mind, Cognitions and Society in the Internet Age. IOS Press, Amsterdam, 10.

[101]

C. Wienrich, K. Schindler, N. Döllinqer, S. Kock, and O. Traupe. 2018. Social Presence and Cooperation in Large-Scale Multi-User Virtual Reality - The Relevance of Social Interdependence for Location-Based Environments. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 207--214.

[102]

Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 7, 3 (June 1998), 225--240.

Digital Library

[103]

Pavel Zahorik and Rick L. Jenison. 1998. Presence as Being-in-the-World. Presence: Teleoperators and Virtual Environments 7, 1 (Feb. 1998), 78--89.

Digital Library

[104]

P. Zimmons and A. Panter. 2003. The Influence of Rendering Quality on Presence and Task Performance in a Virtual Environment. In Proceedings IEEE Virtual Reality. 293--294.

Digital Library

Cited By

Joo JHan SPark IChung T(2024)Immersive Emotion Analysis in VR Environments: A Sensor-Based Approach to Prevent DistortionElectronics10.3390/electronics1308149413:8(1494)Online publication date: 14-Apr-2024
https://doi.org/10.3390/electronics13081494
Tadayyoni HRamirez Campos MQuevedo AMurphy B(2024)Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning ApproachBrain Sciences10.3390/brainsci1405047014:5(470)Online publication date: 7-May-2024
https://doi.org/10.3390/brainsci14050470
Küntzer LSchwab SSpaderna HRock G(2024)ROVER: A Standalone Overlay Tool for Questionnaires in Virtual RealityCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661328(31-39)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661328
Show More Cited By

Index Terms

Breaking The Experience: Effects of Questionnaires in VR User Studies
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Examining Design Choices of Questionnaires in VR User Studies
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Questionnaires are among the most common research tools in virtual reality (VR) user studies. Transitioning from virtuality to reality for giving self-reports on VR experiences can lead to systematic biases. VR allows to embed questionnaires into the ...
Effects on User Experience in an Edutainment Virtual Environment: Comparison Between CAVE and HMD
ECCE '17: Proceedings of the European Conference on Cognitive Ergonomics

The user experience in immersive virtual environment can be broadly defined by the feeling of presence and immersion in a virtual environment. The propensity of feeling "there" and "enveloped by" in the virtual environment seems to be essential in ...
Estimating VR Sickness and user experience using different HMD technologies: An evaluation study
Abstract
This paper presents results of a user study of the effects of virtual reality technology on VR Sickness and User Experience. In our study the participants watched two different panoramic (360) videos, one with relaxing content (beach ...
Highlights
- Assessing VR Sickness discomfort levels using the SSQ and Questionnaire.
- User ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Horizon 2020
DFG

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

66
Total Citations
View Citations
2,817
Total Downloads

Downloads (Last 12 months)375
Downloads (Last 6 weeks)70

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Joo JHan SPark IChung T(2024)Immersive Emotion Analysis in VR Environments: A Sensor-Based Approach to Prevent DistortionElectronics10.3390/electronics1308149413:8(1494)Online publication date: 14-Apr-2024
https://doi.org/10.3390/electronics13081494
Tadayyoni HRamirez Campos MQuevedo AMurphy B(2024)Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning ApproachBrain Sciences10.3390/brainsci1405047014:5(470)Online publication date: 7-May-2024
https://doi.org/10.3390/brainsci14050470
Küntzer LSchwab SSpaderna HRock G(2024)ROVER: A Standalone Overlay Tool for Questionnaires in Virtual RealityCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661328(31-39)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661328
Young JFerreira JPantidi N(2024)"I Shot the Interviewer!": The Effects of In-VR Interviews on Participant Feedback and RapportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642707(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642707
Venkatraj KMeijer WPerusquia-Hernandez MHuisman GEl Ali A(2024)ShareYourReality: Investigating Haptic Feedback and Agency in Virtual Avatar Co-embodimentProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642425(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642425
Wang PMiller MQueiroz ABailenson J(2024)Socially Late, Virtually Present: The Effects of Transforming Asynchronous Social Interactions in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642244(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642244
Chandio YInterrante VAnwar F(2024)Human Factors at Play: Understanding the Impact of Conditioning on Presence and Reaction Time in Mixed RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337212030:5(2400-2410)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372120
Chandio YBashir NInterrante VAnwar F(2024)Investigating the Correlation Between Presence and Reaction Time in Mixed RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.331956330:9(5976-5992)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3319563
Küntzer LSchwab SSpaderna HRock G(2024)Measuring User Experience of Older Adults during Virtual Reality Exergaming2024 16th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX61742.2024.10598263(153-159)Online publication date: 18-Jun-2024
https://doi.org/10.1109/QoMEX61742.2024.10598263
Li YHogue A(2024)StudyVR: A Framework for Streamlining VR User Study Design2024 IEEE Gaming, Entertainment, and Media Conference (GEM)10.1109/GEM61861.2024.10585501(1-5)Online publication date: 5-Jun-2024
https://doi.org/10.1109/GEM61861.2024.10585501
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents