Article

Embodiment and multimodality

Author:

Francis QuekAuthors Info & Claims

ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

Pages 388 - 390

https://doi.org/10.1145/1180995.1181067

Published: 02 November 2006 Publication History

Abstract

Students who are blind are typically one to three years behind their seeing counterparts in mathematics and science. We posit that a key reason for this resides in the inability of such students to access multimodal embodied communicative behavior of mathematics instructors. This impedes the ability of blind students and their teachers to maintain situated communication. In this paper, we set forth the relevant phenomenological analyses to support this claim. We show that mathematical communication and instruction are inherent embodied; that the blind are able to conceptualize visuo-spatial information; and argue that uptake of embodied behavior is critical to receiving relevant mathematical information. Based on this analysis, we advance an approach to provide students who are blind with awareness of their teachers' deictic gestural activity via a set of haptic output devices. We lay forth a set of open research question that researcher in multimodal interfaces may address.

References

[1]

Williams, J. M., Nationwide shortage of teachers for blind students must be corrected. National Federation of the Blind: Advocates for Equality, Canadian Blind Monitor, 2002. 14.

[2]

McNeill, D. and S. Duncan, Growth points in thinking-for-speaking, in Language & Gesture, D. McNeill, Editor. 2000, Cambridge University Press: Cambridge, MA. p. 141--161.

[3]

McNeill, D., Gesture and language dialectic. Acta Linguistica Hafniensia, 2002. 34: p. 7--37.

[4]

Beattie, G. and H. Shovelton, Iconic hand gestures and the predictability of words in context in spontaneous speech. British Journal of Psychology, 2000. 91: p. 473--491.

[5]

Beattie, G., Visible Thought: The New Psychology of body Language. 2003, Hove: Routledge.

[6]

Furuyama, N., Gestural interaction between the instructor and the learner in origami instruction, in Language and gesture. 2000, Cambridge U. Press: Cambridge. p. 99--117.

[7]

Penrose, R., The Emporer's New Mind. 1989, New York: Oxford University Press.

[8]

McNeill, D., Hand and Mind: What Gestures Reveal about thought. 1992, Chicago: University of Chicago Press.

[9]

Smith, N., Gesture and Beyond, in Cognitive Science. 2003, University of California at: Berkeley.

[10]

Haber, R. N., Haber, L. R., Levin, C. A. and Hollyfield, R., Properties of spatial representations: Data from sighted and blind subjects. Percept. & Psychophysics, 1993. 54: p. 1--13.

[11]

Millar, S., Movement cues and body orientation in recall of locations by blind and sighted children. Quarterly Journal of Psychology, 1985(A 37): p. 257--279.

[12]

Landau, B., E. Spelke, and H. Gleitman, Spatial knowledge in a young blind child. Cognition, 1984. 16: p. 225--260.

[13]

Kennedy, J. M., Drawing and the Blind. 1993, New Haven, CT: Yale Press.

[14]

Iverson, J. M. and S. Goldin-Meadow, Why people gesture as they speak. Nature, 1998. 396: p. 228.

[15]

Heller, M. A., Tactile picture perception in sighted and blind people. Behavioural Brain Research, 2002. 135: p. 65--68.

[16]

Craig, R. H., Learning the Nemeth Braille Code: A Manual for Teachers. 1980, Salt Lake City: Brigham Young University Press.

[17]

Eramian, M. G., et al. Talking Tactile Diagrams. in Universal Access in HCI: Inclusive Design in the Information Society, Proceedings of HCI International 2003. 2003. Hersonissos, Crete, Greece.

[18]

Williams, A., E. Kabish, and P. Dourish. From Interaction to Participation: Configuring Space through Embodied Interaction. in Proc. Intl. Conf. Ubiquitous Computing (Ubicomp 2005). 2005. Tokyo, Japan.

Digital Library

[19]

Tall, D., A Theory of Mathematical Growth through Embodiment, Symbolism and Proof, in International Colloquium on Mathematical Learning from Early Childhood to Adulthood. 2005: Nevellis,Belguim.

Cited By

Cheng ZZhao JYang C(2023)Embodied Virtuality: A Method on the Exploration of Sound-Based Virtual JewelleryDesign, User Experience, and Usability10.1007/978-3-031-35696-4_29(401-412)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35696-4_29
Pacheco BSouza-Concilio I(2013)Multimodal interfaces: An enactive approach2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom)10.1109/CogInfoCom.2013.6719302(51-58)Online publication date: Dec-2013
https://doi.org/10.1109/CogInfoCom.2013.6719302

Index Terms

Embodiment and multimodality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models
  2. Interaction design
    1. Interaction design theory, concepts and paradigms

Recommendations

Enabling multimodal communications for enhancing the ability of learning for the visually impaired
ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

Students who are blind are typically one to three years behind their seeing counterparts in mathematics and science. We posit that a key reason for this resides in the inability of such students to access multimodal embodied communicative behavior of ...
Levels of embodiment: linguistic analyses of factors influencing hri
HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

In this paper, we investigate the role of physical embodiment of a robot and its degrees of freedom in HRI. Both factors have been suggested to be relevant in definitions of embodiment, and so far we do not understand their effects on the way people ...
The Role of Embodiment and Simulation in Evaluating HCI: Theory and Framework
Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Human Body, Motion and Behavior
Abstract
In this paper, we argue that embodiment can play an important role in the evaluation of systems developed for Human Computer Interaction. To this end, we describe a simulation platform for building Embodied Human Computer Interactions (EHCI). This ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

November 2006

404 pages

ISBN:159593541X

DOI:10.1145/1180995

General Chairs:
Francis Quek
Virginia Tech, USA
,
Jie Yang
Carnegie Mellon University, USA
,
Program Chairs:
Dominic Massaro
University of California, Santa Cruz, USA
,
Abeer Alwan
University of California, Los Angeles, USA
,
Timothy J. Hazen
Massachusetts Institute of Technology, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI06

Sponsor:

ICMI06: 8th International Conference on Multimodal Interfaces 2006

November 2 - 4, 2006

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
460
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)3

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cheng ZZhao JYang C(2023)Embodied Virtuality: A Method on the Exploration of Sound-Based Virtual JewelleryDesign, User Experience, and Usability10.1007/978-3-031-35696-4_29(401-412)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35696-4_29
Pacheco BSouza-Concilio I(2013)Multimodal interfaces: An enactive approach2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom)10.1109/CogInfoCom.2013.6719302(51-58)Online publication date: Dec-2013
https://doi.org/10.1109/CogInfoCom.2013.6719302

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents