article

Free access

Speech and gestures for graphic image manipulation

Author:

A. G. HauptmannAuthors Info & Claims

ACM SIGCHI Bulletin, Volume 20, Issue SI

Pages 241 - 245

https://doi.org/10.1145/67450.67496

Published: 01 March 1989 Publication History

Abstract

An experiment was conducted with people using gestures and speech to manipulate graphic images on a computer screen. A human was substituted for the recognition devices. The analysis showed that people strongly prefer to use both gestures and speech for the graphics manipulation and that they intuitively use multiple hands and multiple fingers in all three dimensions. There was surprising uniformity and simplicity in the gestures and speech. The analysis of these results provides strong encouragement for future development of integrated multi-modal interaction systems.

References

[1]

Birdwhistell, R.L. Kinesics and context; essays on body motion communication. University of Pennsylvania Press, Philadelphia, 1970.]]

[2]

Dragon Systems, Inc. VoiceScribe-lO00: Speech Recognition System. Newton, MA, 1986.]]

[3]

Ford, W.R., Weeks, G.D. and Chapanis, A. "The effect of serf-imposed brevity on the structure of dyadic communication". Journal of Psychology 104 (1980), 87 - 103.]]

[4]

Friedman, L.A. On the other Hand. Academic Press, New York, 1977.]]

[5]

Fromkin, V. and Rodman, R. An Introduction to Language. Rhinehart and Winston, New York, 1978.]]

Digital Library

[6]

VPL Research, Inc. Dataglove Model 2 System. Redwood City, CA, 1987.]]

[7]

Greene, R. "The drawing prism: A Versatile Graphic input Device". ACM SIGGRAPH '85 19, 3 (1985), 103 - 110.]]

Digital Library

[8]

Hauptmann, A.G., McAvinney, P. and Shepard, S. Gesture Analysis for Graphic Manipulation. Department of Computer Science CMU-CS-88-198, Carnegie-Mellon University, November, 1988.]]

[9]

Hewes, G. "The current Status of the Gestural Theory of Language". Annals of the New York Academy of Science 280 (~976).]]

[10]

Johnstone, E. The Rolky: A Poly-Touch Controller for Electronic Music. ICMC Proceedings, 1985, pp. 291 - 295.]]

[11]

Marlin, J. Design of Man-Computer Dialogues. Prentice-Hall, Englewood Cliffs, 1973.]]

Digital Library

[12]

Lee, S.K., Buxton, W. and Smith, K.C. A Multi-Touch Three Dimensional Touch-Sensitive Tablet. CHI-85 Conference Proceedings: Human Factors in Computing Systems, ACM, San Francisco, CA, April, 1985, pp. 21 - 25.]]

Digital Library

[13]

Myers, J.L. Fundamentals of Experimental Design. Allyn and Bacon, Boston, MA, 1972.]]

[14]

Paget, R. Human Speech. Harcourt, Brace, Jovanovich, New York, 1930.]]

[15]

McAvinney, P. The Sensor Frame - A Gesture-Based Device for the Manipulation of Graphic Objects. Carnegie- Mellon University, 1986.]]

[16]

Shneiderman, B. Software Psychology. Winthrop, Cambridge, 1980.]]

Cited By

Wang NZielasko DMaurer F(2024)User Preferences for Interactive 3D Object Transitions in Cross Reality - An Elicitation StudyProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656698(1-9)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656698
Rodriguez RSullivan BBarrera Machuca MBatmaz ATornatzky COrtega F(2024)An Artists' Perspectives on Natural Interactions for Virtual Reality 3D SketchingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642758(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642758
Grubert J(2023)Mixed Reality Interaction TechniquesSpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_5(109-129)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_5
Show More Cited By

Index Terms

Speech and gestures for graphic image manipulation
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Speech recognition
  2. Computer graphics
    1. Graphics systems and interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices

Recommendations

Speech and gestures for graphic image manipulation
CHI '89: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

An experiment was conducted with people using gestures and speech to manipulate graphic images on a computer screen. A human was substituted for the recognition devices. The analysis showed that people strongly prefer to use both gestures and speech for ...
A graphic object manipulation system using hand gestures
CHI '92: Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems

People use gestures to communitcate with other people without difficutlty. As a method of computer/human interaction, gestures are more intuitive and more direct than existing input devices, such as keyboards, mice, etc., and futhermore, are more ...
Eye-based head gestures
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

A novel method for video-based head gesture recognition using eye information by an eye tracker has been proposed. The method uses a combination of gaze and eye movement to infer head gestures. Compared to other gesture-based methods a major advantage ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCHI Bulletin

ACM SIGCHI Bulletin Volume 20, Issue SI

March 1989

374 pages

ISSN:0736-6906

DOI:10.1145/67450

Editors:
K. Bice
Texas Instruments
,
C. Lewis
Univ. of Colorado, Boulder, CO

Issue’s Table of Contents

CHI '89: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
March 1989
397 pages
ISBN:0897913019
DOI:10.1145/67449
Editors:
K. Bice
Texas Instruments
,
C. Lewis
Univ. of Colorado, Boulder, CO

Copyright © 1989 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 March 1989

Published in SIGCHI Volume 20, Issue SI

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

108
Total Citations
View Citations
1,489
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)15

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang NZielasko DMaurer F(2024)User Preferences for Interactive 3D Object Transitions in Cross Reality - An Elicitation StudyProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656698(1-9)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656698
Rodriguez RSullivan BBarrera Machuca MBatmaz ATornatzky COrtega F(2024)An Artists' Perspectives on Natural Interactions for Virtual Reality 3D SketchingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642758(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642758
Grubert J(2023)Mixed Reality Interaction TechniquesSpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_5(109-129)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-3-030-67822-7_5
Niu HVan Leeuwen CHao JWang GLachmann T(2022)Multimodal Natural Human–Computer Interfaces for Computer-Aided Design: A Review PaperApplied Sciences10.3390/app1213651012:13(6510)Online publication date: 27-Jun-2022
https://doi.org/10.3390/app12136510
Oviatt S(2022)Multimodal Interaction, Interfaces, and AnalyticsHandbook of Human Computer Interaction10.1007/978-3-319-27648-9_22-1(1-29)Online publication date: 1-Apr-2022
https://doi.org/10.1007/978-3-319-27648-9_22-1
Ito ANakada K(2022)User Interface Design Based on Traditional Japanese Air Finger Drawings, “Soragaki”Accentuated Innovations in Cognitive Info-Communication10.1007/978-3-031-10956-0_11(217-237)Online publication date: 27-Sep-2022
https://doi.org/10.1007/978-3-031-10956-0_11
Srinivasan ALee BStasko J(2021)Interweaving Multimodal Interaction With Flexible Unit Visualizations for Data ExplorationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.297805027:8(3519-3533)Online publication date: 1-Aug-2021
https://doi.org/10.1109/TVCG.2020.2978050
Williams AOrtega F(2020)Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained ElicitationProceedings of the ACM on Human-Computer Interaction10.1145/34273304:ISS(1-21)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3427330
Williams AGarcia JOrtega F(2020)Understanding Multimodal User Gesture and Speech Behavior for Object Manipulation in Augmented Reality Using ElicitationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.302356626:12(3479-3489)Online publication date: Dec-2020
https://doi.org/10.1109/TVCG.2020.3023566
Lu YGu CWu LMao J(2020)A 3-D Ray Tracing Model for Short-Range Radar Sensing of Hand Gestures2020 IEEE Asia-Pacific Microwave Conference (APMC)10.1109/APMC47863.2020.9331672(1107-1109)Online publication date: 8-Dec-2020
https://doi.org/10.1109/APMC47863.2020.9331672
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Issue’s Table of Contents