Investigating Pointing Performance for Tangible Surfaces with Physical 3D Targets
Authors: 
Aluna Everitt, 
Anne Roudaut, 
Kasper Hornbæk, 
Mike Fraser, Jason AlexanderAuthors Info & Claims
Aluna Everitt, Anne Roudaut, Kasper Hornbæk, Mike Fraser, Jason AlexanderAuthors Info & ClaimsArticle No.: 583, Pages 617 - 639
Abstract
One of the most fundamental interactions –pointing– is well understood on flat surfaces. However, pointing performance on tangible surfaces with physical targets is still limited for Tangible User Interfaces (TUIs). We investigate the effect of a target’s physical width, height, and distance on user pointing performance. We conducted a study using a reciprocal tapping task (n=19) with physical rods arranged in a circle. We compared our data with five conventional interaction models designed for 2D/3D tasks rather than tangible targets. We show that variance in the movement times was only satisfactorily explained by a model established for volumetric displays (r2=0.954). Analysis shows that movement direction and height should be included as parameters to this model to generalize for 3D tangible targets. Qualitative feedback from participants suggests that pointing at physical targets involves additional human factors (e.g., perception of sharpness or robustness) that need to be investigated further to understand how performance with tangible objects is affected.
References
[1]
Johnny Accot and Shumin Zhai. 2003. Refining Fitts’ law models for bivariate pointing. In Proceedings of the SIGCHI conference on Human factors in computing systems. 193–200.
[2]
Pär-Anders Albinsson and Shumin Zhai. 2003. High precision touch screen interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems. 105–112.
[3]
Jason Alexander, Anne Roudaut, Jürgen Steimle, Kasper Hornbæ k, Miguel Bruns Alonso, Sean Follmer, and Timothy Merritt. 2018. Grand challenges in shape-changing interface research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–14.
[4]
Nicholas T Antony and Peter J Keir. 2010. Effects of posture, movement and hand load on shoulder muscle activity. Journal of Electromyography and Kinesiology, 20, 2 (2010), 191–198.
[5]
Felipe Bacim, Mike Sinclair, and Hrvoje Benko. 2013. Understanding touch selection accuracy on flat and hemispherical deformable surfaces. In Proceedings of Graphics Interface 2013. 197–204.
[6]
Nikola Banovic, Antti Oulasvirta, and Per Ola Kristensson. 2019. Computational Modeling in Human-Computer Interaction. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–7.
[7]
Mayra Donaji Barrera Machuca and Wolfgang Stuerzlinger. 2019. The effect of stereo display deficiencies on virtual hand pointing. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.
[8]
Anil Ufuk Batmaz, Mayra Donaji Barrera Machuca, Junwei Sun, and Wolfgang Stuerzlinger. 2022. The Effect of the Vergence-Accommodation Conflict on Virtual Hand Pointing in Immersive Displays. In CHI Conference on Human Factors in Computing Systems. 1–15.
[9]
Anil Ufuk Batmaz, Mayra Donaji Barrera Machuca, Duc Minh Pham, and Wolfgang Stuerzlinger. 2019. Do head-mounted display stereo deficiencies affect 3D pointing tasks in AR and VR? In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 585–592.
[10]
Gerd Bruder, Frank Steinicke, and Wolfgang Stürzlinger. 2013. Effects of visual conflicts on 3D selection task performance in stereoscopic display environments. In 2013 IEEE Symposium on 3D User Interfaces (3DUI). 115–118.
[11]
Gerd Bruder, Frank Steinicke, and Wolfgang Sturzlinger. 2013. To touch or not to touch? Comparing 2D touch and 3D mid-air interaction on stereoscopic tabletop surfaces. In Proceedings of the 1st symposium on Spatial user interaction. 9–16.
[12]
Yeonjoo Cha and Rohae Myung. 2013. Extended Fitts’ law for 3D pointing tasks using 3D target arrangements. International Journal of Industrial Ergonomics, 43, 4 (2013), 350–355.
[13]
Logan D Clark, Aakash B Bhagat, and Sara L Riggs. 2020. Extending Fitts’ law in three-dimensional virtual environments with current low-cost virtual reality technology. International Journal of Human-Computer Studies, 139 (2020), 102413.
[14]
Aluna Everitt and Jason Alexander. 2017. PolySurface: a design approach for rapid prototyping of shape-changing displays using semi-solid surfaces. In Proceedings of the 2017 Conference on Designing Interactive Systems. 1283–1294.
[15]
Aluna Everitt and Jason Alexander. 2019. 3D Printed Deformable Surfaces for Shape-Changing Displays. Frontiers in Robotics and AI, 6 (2019), 80.
[16]
Aluna Everitt, Alexander Keith Eady, and Audrey Girouard. 2021. Enabling Multi-Material 3D Printing for Designing and Rapid Prototyping of Deformable and Interactive Wearables. In 20th International Conference on Mobile and Ubiquitous Multimedia. 1–11.
[17]
Aluna Everitt, Faisal Taher, and Jason Alexander. 2016. ShapeCanvas: an exploration of shape-changing content generation by members of the public. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems.
[18]
Paul M Fitts. 1954. The information capacity of the human motor system in controlling amplitude of movement. Journal of experimental psychology, 47 (1954).
[19]
Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: dynamic physical affordances and constraints through shape and object actuation. In Uist. 13, 2501988–2502032.
[20]
Jon Froehlich, Jacob O Wobbrock, and Shaun K Kane. 2007. Barrier pointing: using physical edges to assist target acquisition on mobile device touch screens. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility. 19–26.
[21]
Tovi Grossman and Ravin Balakrishnan. 2004. Pointing at trivariate targets in 3D environments. In Proceedings of the SIGCHI conference on Human factors in computing systems. 447–454.
[22]
Yves Guiard, Renaud Blanch, and Michel Beaudouin-Lafon. 2004. Object pointing: a complement to bitmap pointing in GUIs. In Proceedings of Graphics Interface 2004. 9–16.
[23]
Taejin Ha and Woontack Woo. 2010. An empirical evaluation of virtual hand techniques for 3D object manipulation in a tangible augmented reality environment. In 2010 IEEE Symposium on 3D User Interfaces (3DUI). 91–98.
[24]
John Paulin Hansen, Vijay Rajanna, I Scott MacKenzie, and Per Bæ kgaard. 2018. A Fitts’ law study of click and dwell interaction by gaze, head and mouse with a head-mounted display. In Proceedings of the Workshop on Communication by Gaze Interaction. 1–5.
[25]
John Hardy, Christian Weichel, Faisal Taher, John Vidler, and Jason Alexander. 2015. Shapeclip: towards rapid prototyping with shape-changing displays for designers. In Proceedings of the CHI Conference on Human Factors in Computing.
[26]
David M Hoffman, Ahna R Girshick, Kurt Akeley, and Martin S Banks. 2008. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. Journal of vision, 8, 3 (2008), 33–33.
[27]
David Holman and Anne Roudaut. 2013. Simulating Interaction via Computer-Aided Design. In Workshop on Organic Experiences for the CHI Conference 2013.
[28]
David Holman and Roel Vertegaal. 2008. Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM, 51, 6 (2008), 48–55.
[29]
Christian Holz and Patrick Baudisch. 2010. The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
[30]
Eva Hornecker and Jacob Buur. 2006. Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 437–446.
[31]
Hiroshi Ishii. 2008. The tangible user interface and its evolution. Commun. ACM, 51, 6 (2008), 32–36.
[32]
Hiroshi Ishii, Daniel Leithinger, Sean Follmer, Amit Zoran, Philipp Schoessler, and Jared Counts. 2015. TRANSFORM: Embodiment of" Radical Atoms" at Milano Design Week. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. 687–694.
[33]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. 234–241.
[34]
Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper Hornbæ k. 2015. Opportunities and challenges for data physicalization. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3227–3236.
[35]
Izabelle Janzen, Vasanth K Rajendran, and Kellogg S Booth. 2016. Modeling the impact of depth on pointing performance. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 188–199.
[36]
Steven A Jax, David A Rosenbaum, and Jonathan Vaughan. 2007. Extending Fitts’ Law to manual obstacle avoidance. Experimental brain research, 180, 4 (2007).
[37]
Nikhita Joshi and Daniel Vogel. 2019. An Evaluation of Touch Input at the Edge of a Table. In Proceedings of the 2019 CHI Conference on Human Factors in Computing.
[38]
Paul Kabbash and William AS Buxton. 1995. The “prince” technique: Fitts’ law and selection using area cursors. In Proceedings of the SIGCHI conference on Human factors in computing systems. 273–279.
[39]
Regis Kopper, Felipe Bacim, and Doug A Bowman. 2011. Rapid and accurate 3D selection by progressive refinement. In 2011 IEEE Symposium on 3D User Interfaces (3DUI). 67–74.
[40]
Sang-su Lee, Youn-kyung Lim, and Kun-Pyo Lee. 2012. Exploring the effects of size on deformable user interfaces. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services companion.
[41]
Daniel Leithinger, Sean Follmer, Alex Olwal, and Hiroshi Ishii. 2014. Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology. 461–470.
[42]
Daniel Leithinger, Sean Follmer, Alex Olwal, Samuel Luescher, Akimitsu Hogge, Jinha Lee, and Hiroshi Ishii. 2013. Sublimate: state-changing virtual and physical rendering to augment interaction with shape displays. In Proceedings of the SIGCHI conference on human factors in computing systems. 1441–1450.
[43]
Daniel Leithinger, David Lakatos, Anthony DeVincenzi, Matthew Blackshaw, and Hiroshi Ishii. 2011. Direct and gestural interaction with relief: a 2.5 D shape display. In Proceedings of the 24th annual ACM symposium on User interface software and technology. 541–548.
[44]
Paul Lubos, Gerd Bruder, and Frank Steinicke. 2014. Analysis of direct selection in head-mounted display environments. In 2014 IEEE Symposium on 3D User Interfaces (3DUI). 11–18.
[45]
I Scott MacKenzie. 1992. Fitts’ law as a research and design tool in human-computer interaction. Human-computer interaction, 7, 1 (1992), 91–139.
[46]
I Scott MacKenzie and William Buxton. 1992. Extending Fitts’ law to two-dimensional tasks. In Proceedings of the SIGCHI conference on Human factors in computing systems. 219–226.
[47]
Alison McDonald, Bryan R Picco, Alicia L Belbeck, Amy Y Chow, and Clark R Dickerson. 2012. Spatial dependency of shoulder muscle demands in horizontal pushing and pulling. Applied Ergonomics, 43, 6 (2012), 971–978.
[48]
Atsuo Murata and Hirokazu Iwase. 2001. Extending Fitts’ law to a three-dimensional pointing task. Human movement science, 20, 6 (2001), 791–805.
[49]
James Patten, Hiroshi Ishii, Jim Hines, and Gian Pangaro. 2001. Sensetable: a wireless object tracking platform for tangible user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. 253–260.
[50]
Ivan Poupyrev, Tatsushi Nashida, and Makoto Okabe. 2007. Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays. In Proceedings of the 1st international conference on Tangible and embedded interaction.
[51]
Majken K Rasmussen, Esben W Pedersen, Marianne G Petersen, and Kasper Hornbæ k. 2012. Shape-changing interfaces: a review of the design space and open research questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 735–744.
[52]
Simon Robinson, Céline Coutrix, Jennifer Pearson, Juan Rosso, Matheus Fernandes Torquato, Laurence Nigay, and Matt Jones. 2016. Emergeables: Deformable displays for continuous eyes-free mobile interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 3793–3805.
[53]
Anne Roudaut, Abhijit Karnik, Markus Löchtefeld, and Sriram Subramanian. 2013. Morphees: toward high" shape resolution" in self-actuated flexible mobile devices. In Proceedings of SIGCHI Conference on Human Factors in Computing.
[54]
Anne Roudaut, Henning Pohl, and Patrick Baudisch. 2011. Touch input on curved surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing.
[55]
Anne Roudaut, Rebecca Reed, Tianbo Hao, and Sriram Subramanian. 2014. Changibles: analyzing and designing shape changing constructive assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
[56]
Valentin Schwind, Jan Leusmann, and Niels Henze. 2019. Understanding Visual-Haptic Integration of Avatar Hands Using a Fitts’ Law Task in Virtual Reality. In Proceedings of Mensch und Computer 2019. 211–222.
[57]
Orit Shaer and Eva Hornecker. 2010. Tangible user interfaces: past, present, and future directions. Now Publishers Inc.
[58]
R William Soukoreff and I Scott MacKenzie. 2004. Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts’ law research in HCI. International journal of human-computer studies, 61, 6 (2004), 751–789.
[59]
Faisal Taher, John Hardy, Abhijit Karnik, Christian Weichel, Yvonne Jansen, Kasper Hornbæ k, and Jason Alexander. 2015. Exploring interactions with physically dynamic bar charts. In Proceedings of the 33rd annual acm conference on human factors in computing systems. 3237–3246.
[60]
Faisal Taher, Yvonne Jansen, Jonathan Woodruff, John Hardy, Kasper Hornbæ k, and Jason Alexander. 2016. Investigating the use of a dynamic physical bar chart for data exploration and presentation. IEEE transactions on visualization and computer graphics, 23, 1 (2016), 451–460.
[61]
Robert J Teather and Wolfgang Stuerzlinger. 2011. Pointing at 3D targets in a stereo head-tracked virtual environment. In 2011 IEEE Symposium on 3D User Interfaces (3DUI). 87–94.
[62]
Robert J Teather and Wolfgang Stuerzlinger. 2013. Pointing at 3d target projections with one-eyed and stereo cursors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 159–168.
[63]
Eleftherios Triantafyllidis and Zhibin Li. 2021. The challenges in modeling human performance in 3d space with Fitts’ law. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–9.
[64]
Brygg Ullmer and Hiroshi Ishii. 1997. The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software and technology. 223–232.
[65]
Brygg Ullmer and Hiroshi Ishii. 2000. Emerging frameworks for tangible user interfaces. IBM systems journal, 39, 3.4 (2000), 915–931.
[66]
Nout CM van Zon, Clark Borst, Daan M Pool, and Marinus M van Paassen. 2020. Touchscreens for aircraft navigation tasks: comparing accuracy and throughput of three flight deck interfaces using Fitts’ law. Human factors, 62, 6 (2020).
[67]
Jonathan Vaughan, Deborah A Barany, Anthony W Sali, Steven A Jax, and David A Rosenbaum. 2010. Extending Fitts’ Law to three-dimensional obstacle-avoidance movements: support for the posture-based motion planning model. Experimental brain research, 207, 1-2 (2010), 133–138.
[68]
Roel Vertegaal and Ivan Poupyrev. 2008. Organic user interfaces. Commun. ACM, 51, 6 (2008), 26–30.
[69]
Sebastian Vetter, Jennifer Bützler, Nicole Jochems, and Christopher M Schlick. 2011. Fitts’ law in bivariate pointing on large touch screens: Age-differentiated analysis of motion angle effects on movement times and error rates. In International Conference on Universal Access in Human-Computer Interaction.
[70]
Simon Voelker, Christine Sutter, Lei Wang, and Jan Borchers. 2012. Understanding flicking on curved surfaces. In Proceedings of the CHI Conference 2012. 189–198.
[71]
Colin Ware and Kathy Lowther. 1997. Selection using a one-eyed cursor in a fish tank VR environment. ACM Transactions on Computer-Human Interaction (TOCHI), 4, 4 (1997), 309–322.
[72]
Malte Weiss, Simon Voelker, Christine Sutter, and Jan Borchers. 2010. BendDesk: dragging across the curve. In ACM International Conference on Interactive Tabletops and Surfaces. 1–10.
[73]
AT Welford. 1968. Fundamentals of skill (Fundamentals of skill.). New York, NY, US: Methuen.
[74]
Thomas G Whisenand and Henry H Emurian. 1996. Effects of angle of approach on cursor movement with a mouse: Consideration of Fitt’s law. Computers in Human Behavior, 12, 3 (1996), 481–495.
[75]
Difeng Yu, Hai-Ning Liang, Xueshi Lu, Kaixuan Fan, and Barrett Ens. 2019. Modeling endpoint distribution of pointing selection tasks in virtual reality environments. ACM Transactions on Graphics (TOG), 38, 6 (2019), 1–13.
Index Terms
- Investigating Pointing Performance for Tangible Surfaces with Physical 3D Targets
Recommendations
Impact of mobile tangible slider design and its reachability on pointing performance
IHM '16: Actes de la 28ième conference francophone sur l'Interaction Homme-MachineTouch interaction is almost omnipresent on mobile devices and technological advances have made screens bigger. The users can then interact with more content displayed on screen but it is more difficult for the users to reach all the parts of the screen ...
Tangible bots: interaction with active tangibles in tabletop interfaces
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe present interaction techniques for tangible tabletop interfaces that use active, motorized tangibles, what we call Tangible Bots. Tangible Bots can reflect changes in the digital model and assist users by haptic feedback, by correcting errors, by ...
Tangible 3D tabletops: combining tangible tabletop interaction and 3D projection
NordiCHI '12: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through DesignIn this paper we present the tangible 3D tabletop and discuss the design potential of this novel interface. The tangible 3D tabletop combines tangible tabletop interaction with 3D projection in such a way that the tangible objects may be augmented with ...
Comments
Information & Contributors
Information
Published In
December 2022
746 pages
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 November 2022
Published in PACMHCI Volume 6, Issue ISS
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- EPSRC
- EPSRC
- EPSRC
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 535Total Downloads
- Downloads (Last 12 months)188
- Downloads (Last 6 weeks)21
Reflects downloads up to 31 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in