research-article

User-Defined Gestures with Physical Props in Virtual Reality

Authors:

Marco Moran-Ledesma,

Oliver Schneider,

Mark HancockAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 5, Issue ISS

Article No.: 488, Pages 1 - 23

https://doi.org/10.1145/3486954

Published: 05 November 2021 Publication History

Abstract

When interacting with virtual reality (VR) applications like CAD and open-world games, people may want to use gestures as a means of leveraging their knowledge from the physical world. However, people may prefer physical props over handheld controllers to input gestures in VR. We present an elicitation study where 21 participants chose from 95 props to perform manipulative gestures for 20 CAD-like and open-world game-like referents. When analyzing this data, we found existing methods for elicitation studies were insufficient to describe gestures with props, or to measure agreement with prop selection (i.e., agreement between sets of items). We proceeded by describing gestures as context-free grammars, capturing how different props were used in similar roles in a given gesture. We present gesture and prop agreement scores using a generalized agreement score that we developed to compare multiple selections rather than a single selection. We found that props were selected based on their resemblance to virtual objects and the actions they afforded; that gesture and prop agreement depended on the referent, with some referents leading to similar gesture choices, while others led to similar prop choices; and that a small set of carefully chosen props can support multiple gestures.

References

[1]

Parastoo Abtahi and Sean Follmer. 2018. Visuo-Haptic Illusions for Improving the Perceived Performance of Shape Displays. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. Volume 2018- April. ACM Press, New York, New York, USA, 1--13. http://dl.acm.org/citation.cfm?doid=3173574.3173724.

Digital Library

[2]

Sema Alaçam. 2014. The Many Functions of Hand Gestures While Communicating Spatial Ideas An Empirical Case Study. In Proceedings of the XVIII Conference of the Iberoamerican Society of Digital Graphics - SIGraDi: Design in Freedom number 2006. Editora Edgard Blücher, São Paulo, (December 2014), 106--109. http://www.proceedings. blucher.com.br/article-details/14234.

[3]

Shaikh Shawon Arefin Shimon, Courtney Lutton, Zichun Xu, Sarah Morrison-Smith, Christina Boucher, and Jaime Ruiz. 2016. Exploring Non-touchscreen Gestures for Smartwatches. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, (May 2016), 3822--3833. https://dl.acm.org/doi/10. 1145/2858036.2858385.

Digital Library

[4]

Jatin Arora, Aryan Saini, Nirmita Mehra, Varnit Jain, Shwetank Shrey, and Aman Parnami. 2019. VirtualBricks: Exploring a Scalable, Modular Toolkit for Enabling Physical Manipulation in VR. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19 number Chi. ACM Press, New York, New York, USA, 1--12. http://dl.acm.org/citation.cfm?doid=3290605.3300286.

Digital Library

[5]

Autodesk Help. 2021. About create vr. Accessed May 31, 2021. Autodesk. Retrieved 05/31/2021 from https : / / knowledge.autodesk.com/support/alias-products/learn-explore/caas/CloudHelp/cloudhelp/2020/ENU/Alias- CreateVR/files/Alias-CreateVR-aboutcreatevr-html-html.html.

[6]

Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces - ISS '16. ACM Press, New York, New York, USA, (May 2016), 501--504. http://dl.acm.org/citation.cfm?doid=2992154.2996883.

[7]

Gilles Bailly, Thomas Pietrzak, Jonathan Deber, and Daniel J. Wigdor. 2013. Métamorphe: Augmenting Hotkey Usage with Actuated Keys Gilles. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '13. ACM Press, New York, New York, USA, 563. http://dl.acm.org/citation.cfm?doid=2470654.2470734.

[8]

Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. Normaltouch and texturetouch: high-fidelity 3d haptic shape rendering on handheld virtual reality controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). Association for Computing Machinery, Tokyo, Japan, 717--728. https://doi.org/10.1145/2984511.2984526.

Digital Library

[9]

Sarah Buchanan, Bourke Floyd, Will Holderness, and Joseph J. LaViola. 2013. Towards user-defined multi-touch gestures for 3D objects. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces - ITS '13. ACM Press, New York, New York, USA, 231--240. http://dl.acm.org/citation.cfm?doid=2512349.2512825.

[10]

Jeff Butterworth, Andrew Davidson, Stephen Hench, and Marc. T. Olano. 1992. 3DM: A Three Dimensional Modeler Using a Head-Mounted Display. In Proceedings of the 1992 symposium on Interactive 3D graphics - SI3D '92. Volume Part F1296. ACM Press, New York, New York, USA, 135--138. http://portal.acm.org/citation.cfm?doid=147156.147182.

Digital Library

[11]

Jack Shen-Kuen Chang, Georgina Yeboah, Alison Doucette, Paul Clifton, Michael Nitsche, Timothy Welsh, and Ali Mazalek. 2017. Tasc: combining virtual reality with tangible and embodied interactions to support spatial cognition. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17). Association for Computing Machinery, Edinburgh, United Kingdom, 1239--1251. https://doi.org/10.1145/3064663.3064675.

Digital Library

[12]

Lung-Pan Cheng, Li Chang, Sebastian Marwecki, and Patrick Baudisch. 2018. iTurk: Turning Passive Haptics into Active Haptics by Making Users Reconfigure Props in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. Volume 2018-April. ACM Press, New York, New York, USA, 1--10. http://dl.acm.org/citation.cfm?doid=3173574.3173663.

[13]

Eunjung Choi, Heejin Kim, and Min K. Chung. 2014. A taxonomy and notation method for three-dimensional hand gestures. International Journal of Industrial Ergonomics, 44, 1, (January 2014), 171--188. https://linkinghub.elsevier. com/retrieve/pii/S0169814113001285. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. ISS, Article 488. Publication date: November 2021. 488:20 Marco Moran-Ledesma, Oliver Schneider, & Mark Hancock

[14]

Inrak Choi, Heather Culbertson, Mark R. Miller, Alex Olwal, and Sean Follmer. 2017. Grabity. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, (October 2017), 119--130. https://dl.acm.org/doi/10.1145/3126594.3126599.

[15]

Inrak Choi and Sean Follmer. 2016. Wolverine: a wearable haptic interface for grasping in vr. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16 Adjunct). Association for Computing Machinery, Tokyo, Japan, 117--119. https://doi.org/10.1145/2984751.2985725.

Digital Library

[16]

Inrak Choi, Eyal Ofek, Hrvoje Benko, Mike Sinclair, and Christian Holz. 2018. Claw: a multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, Montreal QC, Canada, 1--13. https://doi.org/10.1145/3173574.3174228.

Digital Library

[17]

John W Creswell and Cheryl N Poth. 2018. Qualitative inquiry and research design: Choosing among five approaches. (Fourth edition). Sage Publications, Thousand Oaks, CA, USA.

[18]

Bruno R. De Araújo, Géry Casiez, Joaquim A. Jorge, and Martin Hachet. 2013. Mockup Builder: Direct 3D Modeling On and Above the Surface in a Continuous Interaction Space. Computers & Graphics, 37, 3, (May 2013), 165--178. https://linkinghub.elsevier.com/retrieve/pii/S0097849312001811.

[19]

Alexandra Delazio, Ken Nakagaki, Roberta L. Klatzky, Scott E. Hudson, Jill Fain Lehman, and Alanson P. Sample. 2018. Force Jacket: Pneumatically-Actuated Jacket for Embodied Haptic Experiences. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. ACM Press, New York, New York, USA, 1--12. http://dl.acm.org/citation.cfm?doid=3173574.3173894.

[20]

Seth M. Feeman, Landon B. Wright, and John L. Salmon. 2018. Exploration and evaluation of CAD modeling in virtual reality. Computer-Aided Design and Applications, 15, 6, (November 2018), 892--904. http://www.cadjournal. net/files/vol_15/Vol15No6.html.

[21]

Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 5, 378--382. http://content.apa.org/journals/bul/76/5/378.

[22]

1986. The ecological approach to visual perception. Lawrence Erlbaum Associates, Hillsdale, NJ, USA. Chapter Eight: The Theory of Affordances, 127--136.

[23]

Antonio Gomes, Lahiru Lakmal Priyadarshana, Aaron Visser, Juan Pablo Carrascal, and Roel Vertegaal. 2018. MagicScroll: A Rollable Display Device with Flexible Screen Real Estate and Gestural Input. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, NY, USA, (September 2018), 1--11. https://dl.acm.org/doi/10.1145/3229434.3229442.

Digital Library

[24]

Xiaochi Gu, Yifei Zhang,Weize Sun, Yuanzhe Bian, Dao Zhou, and Per Ola Kristensson. 2016. Dexmo: An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture and Force Feedback in VR. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, (May 2016), 1991--1995. https://dl.acm.org/doi/10.1145/2858036.2858487.

Digital Library

[25]

Mark Hancock, Thomas ten Cate, Sheelagh Carpendale, and Tobias Isenberg. 2010. Supporting Sandtray Therapy on an Interactive Tabletop. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press, New York, New York, USA, 2133. http://portal.acm.org/citation.cfm?doid=1753326.1753651.

Digital Library

[26]

H. G. Hoffman, A. Hollander, K. Schroder, S. Rousseau, and T. Furness. 1998. Physically Touching and Tasting Virtual Objects Enhances the Realism of Virtual Experiences. Virtual Reality, 3, 4, (December 1998), 226--234. http://link.springer.com/10.1007/BF01408703.

[27]

Linda E Homeyer and Daniel S Sweeney. 2016. Sandtray therapy: A practical manual. Routledge, New York, NY, USA.

[28]

Hsin-yu Huang, Chih-wei Ning, Po-Yao Wang, Jen-Hao Cheng, and Lung-Pan Cheng. 2020. Haptic-go-round: A Surrounding Platform for Encounter-type Haptics in Virtual Reality Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, (April 2020), 1--10. https: //dl.acm.org/doi/10.1145/3313831.3376476.

[29]

Jinmiao Huang and Rahul Rai. 2014. Hand gesture based intuitive CAD interface. In Proceedings of the ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. Volume 1A: 34th Computers and Information in Engineering Conference number June. American Society of Mechanical Engineers, Buffalo, NY, USA, (August 2014), 8 pages. https://asmedigitalcollection.asme.org/IDETC-CIE/proceedings/ IDETC-CIE2014/46285/Buffalo,%20New%20York,%20USA/256676.

[30]

Bret Jackson and Daniel F. Keefe. 2016. Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR. IEEE Transactions on Visualization and Computer Graphics, 22, 4, (April 2016), 1442--1451. https: //ieeexplore.ieee.org/document/7383322/.

Digital Library

[31]

Hessam Jahani and Manolya Kavakli. 2018. Exploring a user-defined gesture vocabulary for descriptive mid-air interactions. Cognition, Technology & Work, 20, 1, (February 2018), 11--22. http://link.springer.com/10.1007/s10111- 017-0444-0. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. ISS, Article 488. Publication date: November 2021. User-Defined Gestures with Physical Props in Virtual Reality 488:21

[32]

Maria Karam and m. c. schraefel m. c. 2005. A Taxonomy of Gestures in Human Computer Interaction. Technical report 261149. University of Southampton, 45 pages. http://eprints.soton.ac.uk/id/eprint/261149.

[33]

D.F. Keefe, R.C. Zeleznik, and D.H. Laidlaw. 2007. Drawing on Air: Input Techniques for Controlled 3D Line Illustration. IEEE Transactions on Visualization and Computer Graphics, 13, 5, (September 2007), 1067--1081. http: //ieeexplore.ieee.org/document/4135646/.

Digital Library

[34]

Sumbul Khan and Bige Tunçer. 2019. Automation in Construction Gesture and speech elicitation for 3D CAD modeling in conceptual design. Automation in Construction, 106, July 2017, 102847. https://doi.org/10.1016/j.autcon. 2019.102847.

[35]

Christian Kray, Daniel Nesbitt, John Dawson, and Michael Rohs. 2010. User-defined gestures for connecting mobile phones, public displays, and tabletops. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '10). Association for Computing Machinery, Lisbon, Portugal, 239--248. https://doi.org/10.1145/1851600.1851640.

Digital Library

[36]

Martin Kuka, Radek Trnka, Ale A. Kubna, and Jií Rika. 2016. Free Associations Mirroring Self- and World- Related Concepts: Implications for Personal Construct Theory, Psycholinguistics and Philosophical Psychology. Frontiers in Psychology, 7, June, (June 2016), 1--13. http://journal.frontiersin.org/Article/10.3389/fpsyg.2016.00981/ abstract.

[37]

Byron Lahey, Audrey Girouard, Winslow Burleson, and Roel Vertegaal. 2011. PaperPhone: Understanding the Use of Bend Gestures in Mobile Devices with Flexible Electronic Paper Displays. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11. ACM Press, New York, New York, USA, 1303. http: //dl.acm.org/citation.cfm?doid=1978942.1979136.

Digital Library

[38]

Jaeyeon Lee, Mike Sinclair, Mar Gonzalez-Franco, Eyal Ofek, and Christian Holz. 2019. TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, New York, USA, 1--13. http://dl.acm.org/citation. cfm?doid=3290605.3300301.

Digital Library

[39]

John C. McClelland, Robert J. Teather, and Audrey Girouard. 2017. Haptobend: shape-changing passive haptic feedback in virtual reality. In Proceedings of the 5th Symposium on Spatial User Interaction (SUI '17). Association for Computing Machinery, Brighton, United Kingdom, 82--90. https://doi.org/10.1145/3131277.3132179.

Digital Library

[40]

Tim McGraw, Esteban Garcia, and Drew Sumner. 2017. Interactive swept surface modeling in virtual reality with motion-tracked controllers. In Proceedings of the Symposium on Sketch-Based Interfaces and Modeling number 1. ACM, New York, NY, USA, (July 2017), 1--9. https://dl.acm.org/doi/10.1145/3092907.3092908.

Digital Library

[41]

Mark Mine, Arun Yoganandan, and Dane Coffey. 2014. Making VRWork: Building a Real-World Immersive Modeling Application in the Virtual World. In Proceedings of the 2nd ACM symposium on Spatial user interaction. ACM, New York, NY, USA, (October 2014), 80--89. https://dl.acm.org/doi/10.1145/2659766.2659780.

Digital Library

[42]

Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, C. Schraefel, and Jacob O. Wobbrock. 2014. Reducing legacy bias in gesture elicitation studies. interactions, 21, 3, (May 2014), 40--45. http://dl.acm.org/citation.cfm?doid=2608008.2591689.

Digital Library

[43]

Thomas Muender, Anke V Reinschluessel, Sean Drewes, Dirk Wenig, Tanja Döring, and Rainer Malaka. 2019. Does It Feel Real? Using Tangibles with Different Fidelities to Build and Explore Scenes in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, New York, USA, 1--12. http://dl.acm.org/citation.cfm?doid=3290605.3300903.

[44]

Miguel A Nacenta, Yemliha Kamber, Yizhou Qiang, and Per Ola Kristensson. 2013. Memorability of pre-designed and user-defined gesture sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '13. ACM Press, New York, New York, USA, 1099. http://dl.acm.org/citation.cfm?doid=2470654.2466142.

Digital Library

[45]

Vijayakumar Nanjappan, Hai-Ning Liang, Feiyu Lu, Konstantinos Papangelis, Yong Yue, and Ka Lok Man. 2018. User-elicited dual-hand interactions for manipulating 3d objects in virtual reality environments. Hum.-Centric Comput. Inf. Sci., 8, 1, Article 154, (December 2018), 16 pages. https://doi.org/10.1186/s13673-018-0154--5.

[46]

Donald A. Norman. 2002. The Design of Everyday Things. Basic Books, New York, NY, USA, 257.

Digital Library

[47]

Erica L. Olmsted-Hawala, Elizabeth D. Murphy, Sam Hawala, and Kathleen T. Ashenfelter. 2010. Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). Association for Computing Machinery, Atlanta, Georgia, USA, 2381--2390. https://doi.org/10.1145/1753326.1753685.

Digital Library

[48]

Claudio Pacchierotti, Stephen Sinclair, Massimiliano Solazzi, Antonio Frisoli, Vincent Hayward, and Domenico Prattichizzo. 2017. Wearable Haptic Systems for the Fingertip and the Hand: Taxonomy, Review, and Perspectives. IEEE Transactions on Haptics, 10, 4, (October 2017), 580--600. https://ieeexplore.ieee.org/document/7922602/.

Digital Library

[49]

J. Perret and E. Vander Poorten. 2018. Touching virtual reality: a review of haptic gloves. In ACTUATOR 2018; 16th International Conference on New Actuators. VDE, Bremen, Germany, 270--274. https://ieeexplore.ieee.org/document/ 8470813. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. ISS, Article 488. Publication date: November 2021. 488:22 Marco Moran-Ledesma, Oliver Schneider, & Mark Hancock

[50]

Thammathip Piumsomboon, Adrian Clark, Mark Billinghurst, and Andy Cockburn. 2013. User-defined gestures for augmented reality. In CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13. Volume 2013-April. ACM Press, New York, New York, USA, 955. http://dl.acm.org/citation.cfm?doid=2468356. 2468527.

Digital Library

[51]

Francis Quek, David McNeill, Robert Bryll, Susan Duncan, Xin-Feng Ma, Cemil Kirbas, Karl E. McCullough, and Rashid Ansari. 2002. Multimodal human discourse: gesture and speech. ACM Trans. Comput.-Hum. Interact., 9, 3, (September 2002), 171--193. https://doi.org/10.1145/568513.568514.

Digital Library

[52]

M. H. Quenouille. 1949. Problems in Plane Sampling. The Annals of Mathematical Statistics, 20, 3, (September 1949), 355--375. http://projecteuclid.org/euclid.aoms/1177729989.

[53]

Patrick Reipschläger and Raimund Dachselt. 2019. DesignAR: Immersive 3D-Modeling Combining Augmented Reality with Interactive Displays. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces. ACM, New York, NY, USA, (November 2019), 29--41. https://dl.acm.org/doi/10.1145/3343055.3359718.

Digital Library

[54]

Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). Association for Computing Machinery, Vancouver, BC, Canada, 197--206. https://doi.org/10.1145/1978942.1978971.

Digital Library

[55]

Jaime Ruiz and Daniel Vogel. 2015. Soft-Constraints to Reduce Legacy and Performance Bias to Elicit Wholebody Gestures with Low Arm Fatigue. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15. Volume 2015-April. ACM Press, New York, New York, USA, 3347--3350. http: //dl.acm.org/citation.cfm?doid=2702123.2702583.

[56]

Peter Schulz, Dmitry Alexandrovsky, Felix Putze, Rainer Malaka, and Johannes Schöning. 2019. The Role of Physical Props in VR Climbing Environments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19 number Chi. ACM Press, New York, New York, USA, 1--13. http://dl.acm.org/citation.cfm?doid= 3290605.3300413.

Digital Library

[57]

William A. Scott. 1955. Reliability of Content Analysis: The Case of Nominal Scale Coding. Public Opinion Quarterly, 19, 3, 321. https://academic.oup.com/poq/article-lookup/doi/10.1086/266577.

[58]

Jotaro Shigeyama, Takeru Hashimoto, Shigeo Yoshida, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2019. Transcalibur: AWeight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, New York, USA, 1--11. http://dl.acm.org/citation.cfm?doid=3290605.3300241.

Digital Library

[59]

Adalberto L. Simeone, Eduardo Velloso, and Hans Gellersen. 2015. Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15. Volume 2015-April. ACM Press, New York, New York, USA, 3307--3316. http://dl.acm.org/citation.cfm?doid=2702123.2702389.

Digital Library

[60]

Mike Sinclair, Eyal Ofek, Mar Gonzalez-Franco, and Christian Holz. 2019. CapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, (October 2019), 815--829. https://dl.acm.org/doi/10.1145/3332165.3347891.

Digital Library

[61]

Alexa F. Siu, Eric J. Gonzalez, Shenli Yuan, Jason B. Ginsberg, and Sean Follmer. 2018. shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. Volume 2018-April. ACM Press, New York, New York, USA, 1--13. http://dl.acm.org/citation.cfm?doid=3173574.3173865.

Digital Library

[62]

Evan Strasnick, Christian Holz, Eyal Ofek, Mike Sinclair, and Hrvoje Benko. 2018. Haptic links: bimanual haptics for virtual reality using variable stiffness actuation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, Montreal QC, Canada, 1--12. https://doi.org/ 10.1145/3173574.3174218.

Digital Library

[63]

Keisuke Suzuki, Sohei Wakisaka, and Naotaka Fujii. 2012. Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality. Scientific Reports, 2, 1, (December 2012), 459. http://www.nature.com/ articles/srep00459.

[64]

Shan-yuan Teng, Cheng-Lung Lin, Chi-huan Chiang, Tzu-Sheng Kuo, Liwei Chan, Da-Yuan Huang, and Bing-Yu Chen. 2019. TilePoP: Tile-type Pop-up Prop for Virtual Reality. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, (October 2019), 639--649. https://dl.acm.org/ doi/10.1145/3332165.3347958.

[65]

Theophanis Tsandilas. 2018. Fallacies of Agreement: A Critical Review of Consensus Assessment Methods for Gesture Elicitation. ACM Transactions on Computer-Human Interaction, 25, 3, (June 2018), 1--49. https://dl.acm.org/ doi/10.1145/3182168.

Digital Library

[66]

Consuelo Valdes, Diana Eastman, Casey Grote, Shantanu Thatte, Orit Shaer, Ali Mazalek, BryggUllmer, and Miriam K. Konkel. 2014. Exploring the design space of gestural interaction with active tokens through user-defined Proc. ACM Hum.-Comput. Interact., Vol. 5, No. ISS, Article 488. Publication date: November 2021. User-Defined Gestures with Physical Props in Virtual Reality 488:23 gestures. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14. ACM Press, New York, New York, USA, 4107--4116. http://dl.acm.org/citation.cfm?doid=2556288.2557373.

[67]

Radu-Daniel Vatavu. 2012. User-Defined Gestures for Free-Hand TV Control. In Proceedings of the 10th European conference on Interactive tv and video - EuroiTV '12. ACM Press, New York, New York, USA, 45. http://dl.acm.org/ citation.cfm?doid=2325616.2325626.

Digital Library

[68]

Radu-Daniel Vatavu and Jacob O. Wobbrock. 2015. Formalizing agreement analysis for elicitation studies: new measures, significance test, and toolkit. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, Seoul, Republic of Korea, 1325--1334. https://doi.org/10.1145/2702123.2702223.

Digital Library

[69]

Christian Weichel, Manfred Lau, David Kim, Nicolas Villar, and Hans W. Gellersen. 2014. Mixfab: a mixed-reality environment for personal fabrication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). Association for Computing Machinery, Toronto, Ontario, Canada, 3855--3864. https://doi.org/10. 1145/2556288.2557090.

Digital Library

[70]

Eric Whitmire, Hrvoje Benko, Christian Holz, Eyal Ofek, and Mike Sinclair. 2018. Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfgurable Virtual Reality Controller. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. Volume 2018-April. ACM, New York, NY, USA, (April 2018), 1--4. https://dl.acm.org/doi/10.1145/3170427.3186515.

[71]

Jacob O Wobbrock, Meredith Ringel Morris, and Andrew D Wilson. 2009. User-Defined Gestures for Surface Computing. In Proceedings of the 27th international conference on Human factors in computing systems - CHI 09. Volume 8. ACM Press, New York, New York, USA, 1083. http://dl.acm.org/citation.cfm?doid=1518701.1518866.

Digital Library

[72]

Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the Guessability of Symbolic Input. In CHI '05 extended abstracts on Human factors in computing systems - CHI '05. ACM Press, New York, New York, USA, 1869. http://portal.acm.org/citation.cfm?doid=1056808.1057043.

[73]

Yukang Yan, Chun Yu, Xiaojuan Ma, Xin Yi, Ke Sun, and Yuanchun Shi. 2018. Virtualgrasp: leveraging experience of interacting with physical objects to facilitate digital object retrieval. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, Montreal QC, Canada, 1--13. https://doi.org/10.1145/3173574.3173652.

Digital Library

[74]

Jackie (Junrui) Yang, Hiroshi Horii, Alexander Thayer, and Rafael Ballagas. 2018. Vr grabbers: ungrounded haptic retargeting for precision grabbing tools. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). Association for Computing Machinery, Berlin, Germany, 889--899. https: //doi.org/10.1145/3242587.3242643.

Digital Library

[75]

Andre Zenner and Antonio Kruger. 2017. Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality. IEEE Transactions on Visualization and Computer Graphics, 23, 4, (April 2017), 1285--1294. https://ieeexplore.ieee.org/document/7833030/.

Digital Library

[76]

André Zenner and Antonio Krüger. 2019. Drag:on-- A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, New York, USA, 1--12. http://dl.acm.org/citation.cfm?doid=3290605.3300441.

[77]

Kening Zhu, Taizhou Chen, Feng Han, and Yi-Shiun Wu. 2019. HapTwist: Creating Interactive Haptic Proxies in Virtual Reality Using Low-cost Twistable Artefacts. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, New York, USA, 1--13. http://dl.acm.org/citation.cfm?doid= 3290605.3300923.

Digital Library

Cited By

Degraen DSpeicher MZenner AKrüger A(2024)Exploring User-Defined Interactions for Virtual RealityProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3688882(1-3)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3688882
Greenslade MClark ALukosch S(2024)Becoming Q: Using Design Workshops to Explore Everyday Objects as Interaction Devices for an Augmented Reality Spy GameProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682096(1-11)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682096
Tian JWang YYu KXu LXie JLi FNiu YFan M(2024)Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642884(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642884
Show More Cited By

Index Terms

User-Defined Gestures with Physical Props in Virtual Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
  2. Interaction design
    1. Empirical studies in interaction design

Recommendations

Enabling Reusable Haptic Props for Virtual Reality by Hand Displacement
MuC '21: Proceedings of Mensch und Computer 2021

Virtual Reality (VR) enables compelling visual experiences. However, providing haptic feedback is still challenging. Previous work suggests utilizing haptic props to overcome such limitations and presents evidence that props could function as a single ...
Knock on Wood: Combining Redirected Touching and Physical Props for Tool-Based Interaction in Virtual Reality
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

When physical props serve as proxies for virtual tools used to manipulate the virtual environment, it is challenging to provide appropriate haptic feedback. Redirected tool-mediated manipulation addresses this challenge by distorting the mapping between ...
User-defined gestures for augmented reality
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Recently there has been an increase in research of hand gestures for interaction in the area of Augmented Reality (AR). However this research has focused on developer designed gestures, and little is known about user preference and behavior for gestures ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 5, Issue ISS

ISS

November 2021

481 pages

EISSN:2573-0142

DOI:10.1145/3498314

Editor:
Jeff Nichols
Google

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2021

Published in PACMHCI Volume 5, Issue ISS

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSERC Discovery Grant
NSERC Discovery Accelerator Grant

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
476
Total Downloads

Downloads (Last 12 months)111
Downloads (Last 6 weeks)4

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Degraen DSpeicher MZenner AKrüger A(2024)Exploring User-Defined Interactions for Virtual RealityProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3688882(1-3)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3688882
Greenslade MClark ALukosch S(2024)Becoming Q: Using Design Workshops to Explore Everyday Objects as Interaction Devices for an Augmented Reality Spy GameProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682096(1-11)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682096
Tian JWang YYu KXu LXie JLi FNiu YFan M(2024)Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642884(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642884
Greenslade MClark ALukosch S(2023)Using Everyday Objects as Props for Virtual Objects in First Person Augmented Reality Games: An Elicitation StudyProceedings of the ACM on Human-Computer Interaction10.1145/36110527:CHI PLAY(856-875)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3611052
Lu YWang XGong JZhou LGe S(2022)Classification, Application, Challenge, and Future of Midair Gestures in Augmented RealityJournal of Sensors10.1155/2022/32080472022(1-10)Online publication date: 5-Oct-2022
https://doi.org/10.1155/2022/3208047
Aktar FMaurer F(2022)Elicitation of Interaction Techniques with 3D Data Visualizations in Immersive Environment using HMDs2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00053(238-243)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00053

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents