GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of Grasping
Authors: 
Adwait Sharma, 
Alexander Ivanov, Frances Lai, 
Tovi Grossman, Stephanie SantosaAuthors Info & Claims
Adwait Sharma, Alexander Ivanov, Frances Lai, Tovi Grossman, Stephanie SantosaAuthors Info & ClaimsPages 1275 - 1289
Abstract
Objects are indispensable tools in our daily lives. Recent research has demonstrated their potential to act as conduits for digital interactions with microgestures, however, the primary focus was on situations where the hand firmly grasps an object. We introduce GraspUI, an exploratory design space of object-centric gestures within the seven distinct phases of the grasping process, spanning pre-, during, and post-grasp movements. We conducted ideation sessions with mixed-reality designers from industry and academia to explore gesture integration throughout the entire grasping process. The outcome was 38 storyboards envisioning practical applications. To evaluate the design space’s utility, we performed a video-based assessment with end-users. We then implemented an interactive prototype and quantified the overhead cost of performing proposed gestures through a secondary study. Participants reacted positively to gestures and could integrate them into existing usage of objects. To conclude, we highlight technical and usability guidelines for implementing and extending GraspUI systems.
Supplemental Material
MP4 File
GraspUI full video figure
- Download
- 52.91 MB
References
[1]
2015. Paula Zuccotti’s ’every thing we touch’ photo project catalogs daily life in objects. https://www.bloomberg.com/news/articles/2015-11-27/paula-zuccotti-s-every-thing-we-touch-photo-project-catalogs-daily-life-in-objects
[2]
2023. Jacquard by Google - Saint Laurent. https://atap.google.com/jacquard/products/ysl/
[3]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/
[4]
Fraser Anderson, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. 2015. Supporting Subtlety with Deceptive Devices and Illusory Interactions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). https://doi.org/10.1145/2702123.2702336
[5]
Leonardo Angelini, Francesco Carrino, Stefano Carrino, Maurizio Caon, Omar Abou Khaled, Jürgen Baumgartner, Andreas Sonderegger, Denis Lalanne, and Elena Mugellini. 2014. Gesturing on the Steering Wheel: a User-elicited taxonomy. In Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications(AutomotiveUI ’14). https://doi.org/10.1145/2667317.2667414
[6]
Michelle Annett, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. 2011. Medusa: a proximity-aware multi-touch tabletop. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology(UIST ’11). https://doi.org/10.1145/2047196.2047240
[7]
Michael A Arbib. 2011. Perceptual Structures and Distributed Motor Control. Wiley Online Library. https://doi.org/10.1002/cphy.cp010233
[8]
Tal August, Shamsi Iqbal, Michael Gamon, and Mark Encarnación. 2020. Characterizing the Mobile Microtask Writing Process. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services(MobileHCI ’20). https://doi.org/10.1145/3379503.3403541
[9]
Nikola Banovic, Christina Brant, Jennifer Mankoff, and Anind Dey. 2014. ProactiveTasks: the short of mobile device use sessions. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services(MobileHCI ’14). https://doi.org/10.1145/2628363.2628380
[10]
Sandra Bardot, Surya Rawat, Duy Thai Nguyen, Sawyer Rempel, Huizhe Zheng, Bradley Rey, Jun Li, Kevin Fan, Da-Yuan Huang, Wei Li, and Pourang Irani. 2021. ARO: Exploring the Design of Smart-Ring Interactions for Encumbered Hands. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction(MobileHCI ’21). https://doi.org/10.1145/3447526.3472037
[11]
Iason Batzianoulis, Sahar El-Khoury, Elvira Pirondini, Martina Coscia, Silvestro Micera, and Aude Billard. 2017. EMG-based decoding of grasp gestures in reaching-to-grasping motions. Robotics and Autonomous Systems 91 (2017). https://doi.org/10.1016/j.robot.2016.12.014
[12]
Iason Batzianoulis, Nili E Krausz, Ann M Simon, Levi Hargrove, and Aude Billard. 2018. Decoding the grasping intention from electromyography during reaching motions. Journal of Neuroengineering and Rehabilitation 15, 1 (2018). https://doi.org/10.1186/s12984-018-0396-5
[13]
Iason Batzianoulis, Ann M Simon, Levi Hargrove, and Aude Billard. 2019. Reach-to-grasp motions: Towards a dynamic classification approach for upper-limp prosthesis. In 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE. https://doi.org/10.1109/NER.2019.8717110
[14]
Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic. 2013. Data-Driven Grasp Synthesis—A Survey. IEEE Transactions on Robotics 30, 2 (2013), 289–309. https://doi.org/10.1109/TRO.2013.2289018
[15]
Ahmet Börütecene, Idil Bostan, Ekin Akyürek, Alpay Sabuncuoglu, Ilker Temuzkusu, Çaglar Genç, Tilbe Göksun, and Oguzhan Özcan. 2018. Through the Glance Mug: A Familiar Artefact to Support Opportunistic Search in Meetings. In Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction(TEI ’18). https://doi.org/10.1145/3173225.3173236
[16]
Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. 2019. Contactdb: Analyzing and predicting grasp contact via thermal imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR ’19). https://doi.org/10.1109/CVPR.2019.00891
[17]
Alessandro Carfì, Timothy Patten, Yingyi Kuang, Ali Hammoud, Mohamad Alameh, Elisa Maiettini, Abraham Itzhak Weinberg, Diego Faria, Fulvio Mastrogiovanni, Guillem Alenyà, Lorenzo Natale, Véronique Perdereau, Markus Vincze, and Aude Billard. 2021. Hand-Object Interaction: From Human Demonstrations to Robot Manipulation. Frontiers in Robotics and AI (2021). https://doi.org/10.3389/frobt.2021.714023
[18]
Umberto Castiello. 2005. The neuroscience of grasping. Nature Reviews Neuroscience 6, 9 (2005). https://doi.org/10.1038/nrn1744
[19]
Adrien Chaffangeon Caillet, Alix Goguey, and Laurence Nigay. 2023. µGlyph: a Microgesture Notation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI ’23). https://doi.org/10.1145/3544548.3580693
[20]
Michael Chamunorwa, Mikołaj P Wozniak, Sarah Vöge, Heiko Müller, and Susanne CJ Boll. 2022. Interacting with Rigid and Soft Surfaces for Smart-Home Control. Proceedings of the ACM on Human-Computer Interaction 6, MHCI (2022). https://doi.org/10.1145/3546746
[21]
Justin Cheng, Jaime Teevan, Shamsi T Iqbal, and Michael S Bernstein. 2015. Break It Down: A Comparison of Macro- and Microtasks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). https://doi.org/10.1145/2702123.2702146
[22]
Victor Cheung and Alissa N Antle. 2020. Tangible Interfaces and Interactions in Sci-Fi Movies: A Glimpse at the Possible Future of TUIs through Fictional Tangible Systems. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction(TEI ’20). https://doi.org/10.1145/3374920.3374942
[23]
Kabdo Choi, Hyungyu Shin, Meng Xia, and Juho Kim. 2022. AlgoSolve: Supporting Subgoal Learning in Algorithmic Problem-Solving with Learnersourced Microtasks. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). https://doi.org/10.1145/3491102.3501917
[24]
Christopher Clarke, Alessio Bellino, Augusto Esteves, Eduardo Velloso, and Hans Gellersen. 2016. TraceMatch: a computer vision technique for user input by tracing of animated controls. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing(UbiComp ’16). https://doi.org/10.1145/2971648.2971714
[25]
Christopher Clarke and Hans Gellersen. 2017. MatchPoint: Spontaneous Spatial Coupling of Body Movement for Touchless Pointing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology(UIST ’17). https://doi.org/10.1145/3126594.3126626
[26]
Vanessa Cobus and Wilko Heuten. 2019. To Beep or Not to Beep? Evaluating Modalities for Multimodal ICU Alarms. Multimodal Technologies and Interaction (2019). https://doi.org/10.3390/mti3010015
[27]
Christian Corsten, Ignacio Avellino, Max Möllers, and Jan Borchers. 2013. Instant User Interfaces: Repurposing Everyday Objects as Input Devices. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces(ITS ’13). https://doi.org/10.1145/2512349.2512799
[28]
Enrico Costanza, Samuel A Inverso, Rebecca Allen, and Pattie Maes. 2007. Intimate interfaces in action: Assessing the usability and subtlety of EMG-based motionless gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’07). https://doi.org/10.1145/1240624.1240747
[29]
Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. 2023. ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR ’23). https://doi.org/10.1109/CVPR52729.2023.01244
[30]
Katherine Fennedy, Jeremy Hartmann, Quentin Roy, Simon Tangi Perrault, and Daniel Vogel. 2021. OctoPocus in VR: Using a dynamic guide for 3D mid-air gestures in virtual reality. IEEE Transactions on Visualization and Computer Graphics (2021). https://doi.org/10.1109/TVCG.2021.3101854
[31]
George W Fitzmaurice, Hiroshi Ishii, and William AS Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems(CHI ’95). https://doi.org/10.1145/223904.223964
[32]
Tamar Flash and Neville Hogan. 1985. The Coordination of Arm Movements: An Experimentally Confirmed Mathematical Model. Journal of Neuroscience (1985). https://doi.org/10.1523/jneurosci.05-07-01688.1985
[33]
Weilun Gong, Stephanie Santosa, Tovi Grossman, Michael Glueck, Daniel Clarke, and Frances Lai. 2023. Affordance-Based and User-Defined Gestures for Spatial Tangible Interaction. In Proceedings of the 2023 ACM Designing Interactive Systems Conference(DIS ’23). https://doi.org/10.1145/3563657.3596032
[34]
Tobias Grosse-Puppendahl, Christian Holz, Gabe Cohn, Raphael Wimmer, Oskar Bechtold, Steve Hodges, Matthew S Reynolds, and Joshua R Smith. 2017. Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction. In Proceedings of the 2017 CHI conference on human factors in computing systems(CHI ’17). https://doi.org/10.1145/3025453.3025808
[35]
Kurtis Heimerl, Brian Gawalt, Kuang Chen, Tapan Parikh, and Björn Hartmann. 2012. CommunitySourcing: engaging local crowds to perform expert work via physical kiosks. In Proceedings of the SIGCHI conference on human factors in computing systems(CHI ’12). https://doi.org/10.1145/2207676.2208619
[36]
Steven J Henderson and Steven Feiner. 2008. Opportunistic controls: leveraging natural affordances as tangible user interfaces for augmented reality. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology(VRST ’08). https://doi.org/10.1145/1450579.1450625
[37]
Seongkook Heo, Michelle Annett, Benjamin J Lafreniere, Tovi Grossman, and George W Fitzmaurice. 2017. No Need to Stop What You’re Doing: Exploring No-Handed Smartwatch Interaction. In Proceedings of the 43rd Graphics Interface Conference(GI ’17). https://doi.org/10.20380/GI2017.14
[38]
Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O’Hara, Gavin Smyth, and William Buxton. 2016. Pre-Touch Sensing for Mobile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). https://doi.org/10.1145/2858036.2858095
[39]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems(CHI ’97). https://doi.org/10.1145/258549.258715
[40]
Roland S Johansson and J Randall Flanagan. 2009. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature Reviews Neuroscience 10 (2009). https://doi.org/10.1038/nrn2621
[41]
Lynette A Jones and Susan J Lederman. 2006. Human hand function. Oxford University Press.
[42]
Nikhita Joshi, Parastoo Abtahi, Raj Sodhi, Nitzan Bartov, Jackson Rushing, Christopher Collins, Daniel Vogel, and Michael Glueck. 2023. Transferable Microgestures Across Hand Posture and Location Constraints: Leveraging the Middle, Ring, and Pinky Fingers. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology(UIST ’23). https://doi.org/10.1145/3586183.3606713
[43]
Salman Mohd Khan, Abid Ali Khan, and Omar Farooq. 2021. EMG based classification for pick and place task. Biomedical Physics & Engineering Express 7 (2021). https://doi.org/10.1088/2057-1976/abfa81
[44]
Chang-Min Kim and Tek-Jin Nam. 2022. Exploration on Everyday Objects as an IoT Control Interface. In Proceedings of the 2022 ACM Designing Interactive Systems Conference(DIS ’22). https://doi.org/10.1145/3532106.3533472
[45]
Gierad Laput and Chris Harrison. 2019. Sensing Fine-Grained Hand Activity with Smartwatches. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). https://doi.org/10.1145/3290605.3300568
[46]
Richard T Lauer, P Hunter Peckham, and Kevin L Kilgore. 1999. EEG-based control of a hand grasp neuroprosthesis. Neuroreport 10, 8 (1999). https://doi.org/10.1097/00001756-199906030-00026
[47]
David Ledo, Steven Houben, Jo Vermeulen, Nicolai Marquardt, Lora Oehlberg, and Saul Greenberg. 2018. Evaluation strategies for HCI toolkit research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). https://doi.org/10.1145/3173574.3173610
[48]
Lik-Hang Lee and Pan Hui. 2018. Interaction Methods for Smart Glasses: A Survey. IEEE Access 6 (2018). https://doi.org/10.1109/ACCESS.2018.2831081
[49]
David Lindlbauer, Anna Maria Feit, and Otmar Hilliges. 2019. Context-Aware Online Adaptation of Mixed Reality Interfaces. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST ’19). https://doi.org/10.1145/3332165.3347945
[50]
Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. 2019. MediaPipe: A Framework for Building Perception Pipelines. In Third Workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition (CVPR) 2019. http://arxiv.org/abs/1906.08172
[51]
Kent Lyons, Christopher Skeels, Thad Starner, Cornelis M. Snoeck, Benjamin A. Wong, and Daniel Ashbrook. 2004. Augmenting Conversations Using Dual-Purpose Speech. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology(UIST ’04). https://doi.org/10.1145/1029632.1029674
[52]
Christine L MacKenzie and Thea Iberall. 1994. The grasping hand. Elsevier.
[53]
Ville Mäkelä, Johannes Kleine, Maxine Hood, Florian Alt, and Albrecht Schmidt. 2021. Hidden Interaction Techniques: Concealed Information Acquisition and Texting on Smartphones and Wearables. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). https://doi.org/10.1145/3411764.3445504
[54]
Chiara Martelloni, Jacopo Carpaneto, and Silvestro Micera. 2009. Characterization of EMG patterns from proximal arm muscles during object-and orientation-specific grasps. IEEE Transactions on Biomedical Engineering (2009). https://doi.org/10.1109/TBME.2009.2026470
[55]
Thomas Maskell, Clara Crivellaro, Robert Anderson, Tom Nappey, Vera Araújo-Soares, and Kyle Montague. 2018. Spokespeople: Exploring Routes to Action through Citizen-Generated Data. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). https://doi.org/10.1145/3173574.3173979
[56]
Microsoft. 2023. Hand Interaction Examples. https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk2/features/example-scenes/hand-interaction-examples?view=mrtkunity-2022-05
[57]
Franziska Mueller, Florian Bernard, Oleksandr Sotnychenko, Dushyant Mehta, Srinath Sridhar, Dan Casas, and Christian Theobalt. 2018. GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR ’18). https://handtracker.mpi-inf.mpg.de/projects/GANeratedHands/
[58]
John R Napier. 1956. The Prehensile Movements of the Human Hand. The Journal of Bone and Joint Surgery British Volume 38, 4 (1956). https://doi.org/10.1302/0301-620X.38B4.902
[59]
Henning Pohl and Michael Rohs. 2014. Around-Device Devices: My Coffee Mug is a Volume Dial. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices &; Services(MobileHCI ’14). 81–90. https://doi.org/10.1145/2628363.2628401
[60]
Miya K Rand, YP Shimansky, Abul BMI Hossain, and George E Stelmach. 2008. Quantitative model of transport-aperture coordination during reach-to-grasp movements. Experimental Brain Research 188 (2008). https://doi.org/10.1007/s00221-008-1361-5
[61]
Julie Rico and Stephen Brewster. 2010. Gesture and voice prototyping for early evaluations of social acceptability in multimodal interfaces. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction(ICMI-MLMI ’10). https://doi.org/10.1145/1891903.1891925
[62]
David A Rosenbaum, Ruud J Meulenbroek, Jonathan Vaughan, and Chris Jansen. 2001. Posture-based motion planning: applications to grasping.Psychological review 108 (2001). https://doi.org/10.1037/0033-295x.108.4.709
[63]
Adwait Sharma, Michael A Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, and Jürgen Steimle. 2021. SoloFinger: Robust Microgestures while Grasping Everyday Objects. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). https://doi.org/10.1145/3411764.3445197
[64]
Adwait Sharma, Joan Sol Roo, and Jürgen Steimle. 2019. Grasping Microgestures: Eliciting Single-hand Microgestures for Handheld Objects. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). https://doi.org/10.1145/3290605.3300632
[65]
Roy Shilkrot, Jochen Huber, Jürgen Steimle, Suranga Nanayakkara, and Pattie Maes. 2015. Digital Digits: A Comprehensive Survey of Finger Augmentation Devices. Comput. Surveys (2015). https://doi.org/10.1145/2828993
[66]
Yanke Tan, Sang Ho Yoon, and Karthik Ramani. 2017. BikeGesture: User Elicitation and Performance of Micro Hand Gesture as Input for Cycling. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems(CHI EA ’17). https://doi.org/10.1145/3027063.3053075
[67]
Brandon T Taylor and V Michael Bove Jr. 2009. Graspables: grasp-recognition as a user interface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’09). https://doi.org/10.1145/1518701.1518842
[68]
Khai N. Truong, Thariq Shihipar, and Daniel J. Wigdor. 2014. Slide to X: Unlocking the Potential of Smartphone Unlocking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). https://doi.org/10.1145/2556288.2557044
[69]
Rajan Vaish, Keith Wyngarden, Jingshu Chen, Brandon Cheung, and Michael S. Bernstein. 2014. Twitch Crowdsourcing: Crowd Contributions in Short Bursts of Time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). https://doi.org/10.1145/2556288.2556996
[70]
Vuforia. 2023. VuMarks. https://library.vuforia.com/objects/vumarks
[71]
Stephan Waldert, Hubert Preissl, Evariste Demandt, Christoph Braun, Niels Birbaumer, Ad Aertsen, and Carsten Mehring. 2008. Hand Movement Direction Decoded from MEG and EEG. Journal of Neuroscience 28 (2008). https://doi.org/10.1523/JNEUROSCI.5171-07.2008
[72]
Martin Weigel, Vikram Mehta, and Jürgen Steimle. 2014. More than touch: understanding how people use skin as an input surface for mobile computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’14). https://doi.org/10.1145/2556288.2557239
[73]
Thomas J Williams, Simon L Jones, Christof Lutteroth, Elies Dekoninck, and Hazel C Boyd. 2021. Augmented Reality and Older Adults: A Comparison of Prompting Types. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). https://doi.org/10.1145/3411764.3445476
[74]
Raphael Wimmer. 2010. Grasp sensing for human-computer interaction. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction(TEI ’11). https://doi.org/10.1145/1935701.1935745
[75]
Katrin Wolf, Markus Funk, Pascal Knierim, and Markus Löchtefeld. 2016. Survey of interactive displays through mobile projections. International Journal of Mobile Human Computer Interaction 8 (2016). http://doi.org/10.4018/IJMHCI.2016100102
[76]
Katrin Wolf, Anja Naumann, Michael Rohs, and Jörg Müller. 2011. A taxonomy of microinteractions: Defining microgestures based on ergonomic and scenario-Dependent requirements. In 13th International Conference on Human-Computer Interaction(INTERACT ’11). https://doi.org/10.1007/978-3-642-23774-4_45
[77]
Haijun Xia, Michael Glueck, Michelle Annett, Michael Wang, and Daniel Wigdor. 2022. Iteratively designing gesture vocabularies: A survey and analysis of best practices in the HCI literature. In ACM Transactions on Computer-Human Interaction(TOCHI ’22). https://doi.org/10.1145/3503537
[78]
Xuhai Xu, Anna Yu, Tanya R. Jonker, Kashyap Todi, Feiyu Lu, Xun Qian, João Marcelo Evangelista Belo, Tianyi Wang, Michelle Li, Aran Mun, Te-Yen Wu, Junxiao Shen, Ting Zhang, Narine Kokhlikyan, Fulton Wang, Paul Sorenson, Sophie Kim, and Hrvoje Benko. 2023. XAIR: A Framework of Explainable AI in Augmented Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(CHI ’23). https://doi.org/10.1145/3544548.3581500
[79]
Yang Zhang, Michel Pahud, Christian Holz, Haijun Xia, Gierad Laput, Michael McGuffin, Xiao Tu, Andrew Mittereder, Fei Su, William Buxton, and Ken Hinckley. 2019. Sensing Posture-Aware Pen+Touch Interaction on Tablets. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). https://doi.org/10.1145/3290605.3300285
[80]
Qian Zhou, Sarah Sykes, Sidney Fels, and Kenrick Kin. 2020. Gripmarks: Using Hand Grips to Transform In-Hand Objects into Mixed Reality Input. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). https://doi.org/10.1145/3313831.3376313
[81]
John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems(CHI ’07). https://doi.org/10.1145/1240624.1240704
[82]
Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. 2023. Object Detection in 20 Years: A Survey. Proc. IEEE (2023). https://doi.org/10.1109/JPROC.2023.3238524
Index Terms
- GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of Grasping
Recommendations
Grasping Microgestures: Eliciting Single-hand Microgestures for Handheld Objects
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsSingle-hand microgestures have been recognized for their potential to support direct and subtle interactions. While pioneering work has investigated sensing techniques and presented first sets of intuitive gestures, we still lack a systematic ...
SoloFinger: Robust Microgestures while Grasping Everyday Objects
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsUsing microgestures, prior work has successfully enabled gestural interactions while holding objects. Yet, these existing methods are prone to false activations caused by natural finger movements while holding or manipulating the object. We address this ...
How Do People with Limited Movement Personalize Upper-Body Gestures? Considerations for the Design of Personalized and Accessible Gesture Interfaces
ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and AccessibilityAlways-on, upper-body input from sensors like accelerometers, infrared cameras, and electromyography hold promise to enable accessible gesture input for people with upper-body motor impairments. When these sensors are distributed across the person’s body,...
Comments
Information & Contributors
Information
Published In
July 2024
3616 pages
ISBN:9798400705830
DOI:10.1145/3643834
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
DIS '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,158 of 4,684 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 470Total Downloads
- Downloads (Last 12 months)470
- Downloads (Last 6 weeks)307
Reflects downloads up to 15 Sep 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in