Article

Interacting with large displays from a distance with vision-tracked multi-finger gestural input

Authors:

Abhishek Ranjan,

Ravin BalakrishnanAuthors Info & Claims

UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology

Pages 43 - 52

https://doi.org/10.1145/1095034.1095042

Published: 23 October 2005 Publication History

Abstract

We explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of asymmetric and symmetric interactions, including fast targeting and navigation to all parts of a large display from the comfort of a desk and chair, as well as techniques that exploit the ability of the vision-based hand tracking system to provide multi-finger identification and full 2D hand segmentation. We also posit a design that allows for handling multiple concurrent users.

References

[1]

Balakrishnan, R., and MacKenzie, I. S. (1997). Performance Differences in the Fingers, Wrist, and Forearm in Computer Input Control. In Proceedings of ACM CHI Conference. p. 303--310.

Digital Library

[2]

Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. (2003). Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact. p. 57--64.

[3]

Bezerianos, A and Balakrishnan, R. (2005 -- in press). The Vacuum: Facilitating the Manipulation of Distant Objects. In Proceedings of ACM CHI Conference.

Digital Library

[4]

Bier, E. A., Stone, M. C., Pier, K., Buxton, W., DeRose, T. D. (1993). Toolglass and Magic Lenses: The See-through Interface. In Proceedings of ACM SIGGRAPH. p. 73--80.

Digital Library

[5]

Cao, X., and Balakrishnan, R. (2003). VisionWand: Interaction Techniques for Large Displays Using a Passive Wand Tracked in 3D. In Proceedings of ACM UIST Symposium. p. 173--182.

Digital Library

[6]

Davis, J., Chen, X. (2002). Lumipoint: Multi-User Laser-Based Interaction on Large Tiled Displays. In Displays, Volume 23, Issue 5.

[7]

Dietz, P., and Leigh, D. (2001). DiamondTouch: A Multi-user Touch Technology. In Proceedings of ACM UIST Symposium. p. 219--226.

Digital Library

[8]

Fiala, M. (2004). ARTag Revision 1, A Fiducial Marker System Using Digital Techniques. NRC/ERB-1117 Technical Report. National Research Council of Canada.

[9]

Geißler, J. (1998). Shuffle, Throw or Take It! Working Efficiently with an Interactive Wall. (1998). In ACM CHI Conference, Extended Abstracts. p. 265--266.

Digital Library

[10]

Graham, E., MacKenzie, C. (1996). Physical versus Virtual Pointing. In Proceedings of ACM CHI Conference. p. 292--299.

Digital Library

[11]

Guiard, Y. (1987). Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. In Journal of Motor Behavior, 19, p. 486--517.

[12]

Guimbretière, F., Stone, M., Winograd, T. (2001). Fluid Interaction with High-resolution Wall-size Displays. In Proceedings of ACM UIST Symposium. p. 21--30.

Digital Library

[13]

Hardenberg, C. V., Bérard, F. (2001). Bare-Hand Human-Computer Interaction. In Proceedings of the Workshop on Perceptive User Interface.

Digital Library

[14]

Hinckley, K., Pausch, R., Proffitt, D., Patten, J., Kassell, N. (1997). Cooperative Bimanual Action. In Proceedings of ACM CHI Conference. p. 27--34.

Digital Library

[15]

Humphreys, G., Houston, M., Ng, R., Frank, R., et al. (2002). Chromium: A Stream-Processing Framework for Interactive Rendering on Clusters. In ACM Transactions on Graphics, Proceedings of ACM SIGGRAPH. p. 693--702.

Digital Library

[16]

Izadi, S., Brignull, H., Rodden, T., Rogers, Y., Underwood, M. (2003). Dynamo: A Public Interactive Surface Supporting the Cooperative Sharing and Exchange of Media. In Proceedings of ACM UIST Symposium. p. 159--168.

Digital Library

[17]

Johanson, B., Fox, A., and Winograd, T. (2002). The Interactive Workspace Project: Experiences with Ubiquitous Computing Rooms. In IEEE Pervasive Computing. p. 67--74.

Digital Library

[18]

Johanson, B., Hutchins, G., Winograd, T., Stone, M. (2002). Pointright: Experience with Flexible Input Redirection in Interactive Workspaces. In Proceedings of ACM UIST Symposium. p. 227--234.

Digital Library

[19]

Kabbash, P., Buxton, W., and Sellen, A. (1994). Two-handed Input in a Compound Task. In Proceedings of ACM CHI Conference. p. 17--423.

Digital Library

[20]

Khan, A., Matejka, J., Fitzmaurice, G., Kurtenbach, G. (2005 -- in press). Spotlight: Directing Users' Visual Attention on Large Displays. In Proceedings of the ACM CHI Conference.

Digital Library

[21]

Khan, A., Fitzmaurice, G., Almeida, D., Burtnyk, N., Kurtenbach, G. (2004). A Remote Control Interface for Large Displays. In Proceedings of ACM UIST Symposium. p. 127--136.

Digital Library

[22]

Kirsh, D., & Maglio, P. (1994). On Distinguishing Epistemic from Pragmatic action. In Cognitive Science, 18. p. 513--549.

[23]

Kirstein, C. and Muller, H. (1998). Interaction with a Projection Screen using a Camera-tracked Laser Pointer. In Proceedings of Multimedia Modeling. p. 191.

Digital Library

[24]

Kjeldsen, R., and Hartman, J. (2001). Design Issues for Vision-based Computer Interaction Systems. In Proceedings of the Workshop on Perceptive User Interfaces.

Digital Library

[25]

Kurtenbach, G., Fitzmaurice, G., Baudel, T., Buxton, W. (1997). The Design of a GUI Paradigm based on Tablets, Two Hands, and Transparency. In Proceedings of ACM CHI Conference. p. 35--42.

Digital Library

[26]

Malik, S. and Laszlo, J. (2004). Visual Touchpad: A Two-handed Gestural Input Device. In Proceedings of the ACM International Conference on Multimodal Interfaces. p. 289--296.

Digital Library

[27]

Mynatt, E., Igarashi, T., Edwards, W., LaMarca, A. (1999). Flatland: New Dimensions in Office Whiteboards. In Proceedings of ACM CHI Conference. p. 346--353.

Digital Library

[28]

Pederson, E., McCall, K., Moran, T., Halasz, F. (1993). Tivoli: An Electronic Whiteboard for Informal Workgroup Meetings. In Proceedings of ACM CHI Conference. p. 391--398.

Digital Library

[29]

Rekimoto, J. (2002). SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. In Proceedings of ACM CHI Conference. p. 113--120.

Digital Library

[30]

Swaminathan, K, and Sato, S. (1997). Interaction Design for Large Displays. Interactions, Volume 4, Issue 1.

Digital Library

[31]

Tactiva. (2005). TactaPad. http://www.tactiva.com.

[32]

Wellner, P. (1993). Interacting with Paper on the DigitalDesk. In Communications of the ACM, Volume 36, Issue 7. p. 87--96.

Digital Library

[33]

Wilson, A. (2004). Touchlight: An Imaging Touch Screen and Display for Gesture-Based Interaction. In Proceedings of the International Conference on Multimodal Interfaces. p. 69--76.

Digital Library

[34]

Wu, M., and Balakrishnan, R. (2003). Multi-finger and Whole Hand Gestural Interaction Techniques for Multi-user Tabletop Displays. In Proceedings of ACM UIST Symposium. p. 193--202.

Digital Library

Cited By

Belkacem ITominski CMédoc NKnudsen SDachselt RGhoniem M(2024)Interactive Visualization on Large High‐Resolution Displays: A SurveyComputer Graphics Forum10.1111/cgf.1500143:6Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15001
Boorboor SKim YHu PMoses JColle BKaufman A(2024)Submerse: Visualizing Storm Surge Flooding Simulations in Immersive Display EcologiesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.333251130:9(6365-6377)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3332511
Wang XLiang WYu FHan XNiu HYan X(2024)A Gesture Recognition System Based On MEMS Accelerometer2024 36th Chinese Control and Decision Conference (CCDC)10.1109/CCDC62350.2024.10587430(3987-3992)Online publication date: 25-May-2024
https://doi.org/10.1109/CCDC62350.2024.10587430
Show More Cited By

Index Terms

Interacting with large displays from a distance with vision-tracked multi-finger gestural input
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices
    2. Interaction paradigms

Recommendations

Visual touchpad: a two-handed gestural input device
ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

This paper presents the Visual Touchpad, a low-cost vision-based input device that allows for fluid two-handed interactions with desktop PCs, laptops, public kiosks, or large wall displays. Two downward-pointing cameras are attached above a planar ...
Interacting with large displays from a distance with vision-tracked multi-finger gestural input
SIGGRAPH '06: ACM SIGGRAPH 2006 Sketches

We explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of ...
Multi-finger gestural interaction with 3D volumetric displays
SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

Volumetric displays provide interesting opportunities and challenges for 3D interaction and visualization, particularly when used in a highly interactive manner. We explore this area through the design and implementation of techniques for interactive ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology

October 2005

270 pages

ISBN:1595932712

DOI:10.1145/1095034

General Chairs:
Patrick Baudisch
Microsoft Research
,
Mary Czerwinski
Microsoft Research
,
Program Chair:
Dan Olsen
Brigham Young University, Provo, UT

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

UIST05

Sponsor:

UIST05: The 18th Annual ACM Symposium on User Interface Software and Technology

October 23 - 26, 2005

WA, Seattle, USA

Acceptance Rates

UIST '05 Paper Acceptance Rate 31 of 159 submissions, 19%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

127
Total Citations
View Citations
3,086
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Belkacem ITominski CMédoc NKnudsen SDachselt RGhoniem M(2024)Interactive Visualization on Large High‐Resolution Displays: A SurveyComputer Graphics Forum10.1111/cgf.1500143:6Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15001
Boorboor SKim YHu PMoses JColle BKaufman A(2024)Submerse: Visualizing Storm Surge Flooding Simulations in Immersive Display EcologiesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.333251130:9(6365-6377)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3332511
Wang XLiang WYu FHan XNiu HYan X(2024)A Gesture Recognition System Based On MEMS Accelerometer2024 36th Chinese Control and Decision Conference (CCDC)10.1109/CCDC62350.2024.10587430(3987-3992)Online publication date: 25-May-2024
https://doi.org/10.1109/CCDC62350.2024.10587430
Lou XChen ZHansen PPeng R(2022)Asymmetric Free-Hand Interaction on a Large Display and Inspirations for Designing Natural User InterfacesSymmetry10.3390/sym1405092814:5(928)Online publication date: 2-May-2022
https://doi.org/10.3390/sym14050928
Le KTran TChlasta KKrejtz KFjeld MKunz A(2021)VXSlate: Exploring Combination of Head Movements and Mobile Touch for Large Virtual Display InteractionProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462076(283-297)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3461778.3462076
Almutairi ADavies NMikusz MLangheinrich MClinch SCobus VPrange SClinch SRukzio EGugenheimer J(2020)Designing for conflictProceedings of the 9TH ACM International Symposium on Pervasive Displays10.1145/3393712.3395336(47-54)Online publication date: 4-Jun-2020
https://dl.acm.org/doi/10.1145/3393712.3395336
Liu JZhang HLi C(2020)COMTIS: Customizable touchless interaction system for large screen visualizationVirtual Reality & Intelligent Hardware10.1016/j.vrih.2020.01.0032:2(162-174)Online publication date: Apr-2020
https://doi.org/10.1016/j.vrih.2020.01.003
Schuman CStofer KAnthony LNeff HChang PSoni NDarrow ALuc AMorales AAlexandre JKirkland B(2020)Ocean Data Visualization on a Touchtable Demonstrates Group Content Learning, Science Practices Use, and Potential Embodied CognitionResearch in Science Education10.1007/s11165-020-09951-952:2(445-457)Online publication date: 6-Aug-2020
https://doi.org/10.1007/s11165-020-09951-9
Otto MLampen EAgethen PZachmann GRukzio E(2020)Using large-scale augmented floor surfaces for industrial applications and evaluation on perceived sizesPersonal and Ubiquitous Computing10.1007/s00779-020-01433-z26:3(721-736)Online publication date: 11-Aug-2020
https://doi.org/10.1007/s00779-020-01433-z
Sharma GRadke R(2020)Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as TouchpadsIntelligent Systems and Applications10.1007/978-3-030-55190-2_22(285-302)Online publication date: 25-Aug-2020
https://doi.org/10.1007/978-3-030-55190-2_22
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents