research-article

Visual attention in spoken human-robot interaction

Authors:

Matthew W. CrockerAuthors Info & Claims

HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction

Pages 77 - 84

https://doi.org/10.1145/1514095.1514111

Published: 09 March 2009 Publication History

Abstract

Psycholinguistic studies of situated language processing have revealed that gaze in the visual environment is tightly coupled with both spoken language comprehension and production. It has also been established that interlocutors monitor the gaze of their partners, a phenomenon called "joint attention", as a further means for facilitating mutual understanding. We hypothesise that human-robot interaction will benefit when the robot's language-related gaze behaviour is similar to that of people, potentially providing the user with valuable non-verbal information concerning the robot's intended message or the robot's successful understanding. We report findings from two eye-tracking experiments demonstrating (1) that human gaze is modulated by both the robot speech and gaze, and (2) that human comprehension of robot speech is improved when the robot's real-time gaze behaviour is similar to that of humans.

References

[1]

G. Altmann and Y. Kamide. Now you see it, now you don't: Mediating the mapping between language and the visual world. In J. Henderson and F. Ferreira, editors, The Interface of Language, Vision, and Action: Eye Movements and The Visual World, pages 347--386. Psychology Press, NY, 2004.

[2]

S. Baron-Cohen, D. Baldwin, and M. Crowson. Do Children with Autism Use the Speaker's Direction of Gaze Strategy to Crack the Code of Language? Child Development, 68:48--57, 1997.

[3]

S. Baron-Cohen, A. Leslie, and U. Frith. Does the autistic child have a 'theory of mind'? Cognition, 21:37--46, 1985.

[4]

J. Cassell, O. Torres, and S. Prevost. Turn Taking vs. Discourse Structure: How Best to Model Multimodal Conversation. Machine Conversations, pages 143--154, 1999.

[5]

P. D. Chris Moore, Philip J. Dunham, editor. Joint Attention Its Origins and Role in Development. LEA, 1995.

[6]

Z. M. Griffin. Gaze durations during speech reflect word selection and phonological encoding. Cognition, 82:B1--B14, 2001.

[7]

Z. M. Griffin and K. Bock. What the eyes say about speaking. Psychological Science, 11:274--279, 2000.

[8]

J. Hanna and S. Brennan. Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57:596--615, 2007.

[9]

M. Imai, T. Kanda, T. Ono, H. Ishiguro, and K. Mase. Robot mediated round table: Analysis of the effect of robot's gaze. In Proc. of 11th IEEE ROMAN '02, pages 411--416, 2002.

[10]

S. Kiesler and J. Goetz. Mental models of robotic assistants. In Conference on Human Factors in Computing Systems, pages 576--577, 2002.

Digital Library

[11]

S. Kiesler, A. Powers, S. Fussell, and C. Torrey. Anthropomorphic interactions with a robot and robot-like agent. Social Cognition, 26:169--181, 2008.

[12]

P. Knoeferle and M. W. Crocker. The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking. Cognitive Science, 30:481--529, 2006.

[13]

P. Knoeferle and M. W. Crocker. The influence of recent scene events on spoken comprehension: evidence from eye-movements. Journal of Memory and Language (Special issue: Language-Vision Interaction), 57:519--543, 2007.

[14]

A. Meyer, A. Sleiderink, and W. Levelt. Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66:B25--B33, 1998.

[15]

B. Mutlu, J. Hodgins, and J. Forlizzi. A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior. In Proceedings 2006 IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS'06), Genova, Italy, 2006.

[16]

B. Scassellati. Theory of mind for a humanoid robot. In 1st IEEE/RSJ International Conference on Humanoid Robotics (Humanoids 2000), Cambridge, MA., 2000.

[17]

C. L. Sidner, C. Lee, C. Kidd, N. Lesh, and C. Rich. Explorations in engagement for humans and robots. Artificial Intelligence, 166(1-2):140--164, 2005.

Digital Library

[18]

M. Staudte and M. W. Crocker. The utility of gaze in human-robot interaction. In Proceedings of "Metrics for Human-Robot Interaction", Workshop at ACM/IEEE HRI 2008, Amsterdam, Netherlands, 2008.

[19]

M. K. Tanenhaus, M. Spivey-Knowlton, K. Eberhard, and J. Sedivy. Integration of visual and linguistic information in spoken language comprehension. Science, 268:1632--1634, 1995.

[20]

S. Woods, M. Walters, K. L. Koay, and K. Dautenhahn. Comparing Human Robot Interaction Scenarios Using Live and Video Based Methods: Towards a Novel Methodological Approach. In Proc. AMC'06, The 9th International Workshop on Advanced Motion Control, 2006.

[21]

A. Yamazaki, K. Yamazaki, Y. Kuno, M. Burdelski, M. Kawashima, and H. Kuzuoka. Precision timing in human-robot interaction: Coordination of head movement and utterance. In Proceedings of CHI '08, 2008.

Digital Library

Cited By

Hou YCheon EJung MGrollman DBroadbent EJu WSoh HWilliams T(2024)Power in Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634949(269-282)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634949
Maehigashi ATsumura TYamada S(2024)Impacts of Robot Beep Timings on Trust Dynamics in Human-Robot InteractionInternational Journal of Social Robotics10.1007/s12369-024-01181-716:11-12(2205-2222)Online publication date: 24-Oct-2024
https://doi.org/10.1007/s12369-024-01181-7
Pascher MGruenefeld USchneegass SGerken J(2023)How to Communicate Robot Motion Intent: A Scoping ReviewProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580857(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580857
Show More Cited By

Index Terms

Visual attention in spoken human-robot interaction

Recommendations

Footing in human-robot conversations: how robots might shape participant roles using gaze cues
HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction

During conversations, speakers establish their and others' participant roles (who participates in the conversation and in what capacity)--or "footing" as termed by Goffman-using gaze cues. In this paper, we study how a robot can establish the ...
Spontaneous spoken dialogues with the furhat human-like robot head
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction

Furhat [1] is a robot head that deploys a back-projected animated face that is realistic and human-like in anatomy. Furhat relies on a state-of-the-art facial animation architecture allowing accurate synchronized lip movements with speech, and the ...
On the Benefit of Independent Control of Head and Eye Movements of a Social Robot for Multiparty Human-Robot Interaction
Human-Computer Interaction
Abstract
The human gaze direction is the sum of the head and eye movements. The coordination of these two segments has been studied and models of the contribution of head movement to the gaze of virtual agents or robots have been proposed. However, these ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '09: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction

March 2009

348 pages

ISBN:9781605584041

DOI:10.1145/1514095

General Chairs:
Matthias Scheutz
Indiana University, USA
,
François Michaud
Université de Sherbrooke, Canada
,
Program Chairs:
Pamela Hinds
Stanford University, USA
,
Brian Scassellati
Yale University, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HRI09

Sponsor:

HRI09: International Conference on Human Robot Interaction

March 9 - 13, 2009

California, La Jolla, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

74
Total Citations
View Citations
733
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)3

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hou YCheon EJung MGrollman DBroadbent EJu WSoh HWilliams T(2024)Power in Human-Robot InteractionProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634949(269-282)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634949
Maehigashi ATsumura TYamada S(2024)Impacts of Robot Beep Timings on Trust Dynamics in Human-Robot InteractionInternational Journal of Social Robotics10.1007/s12369-024-01181-716:11-12(2205-2222)Online publication date: 24-Oct-2024
https://doi.org/10.1007/s12369-024-01181-7
Pascher MGruenefeld USchneegass SGerken J(2023)How to Communicate Robot Motion Intent: A Scoping ReviewProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580857(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580857
Somashekarappa VSayeed AHowes C(2023)Neural Network Implementation of Gaze-Target Prediction for Human-Robot Interaction2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309483(2238-2244)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309483
Maehigashi ATsumura TYamada S(2023)Effects of Beep-Sound Timings on Trust Dynamics in Human-Robot InteractionSocial Robotics10.1007/978-3-031-24670-8_57(652-662)Online publication date: 2-Feb-2023
https://doi.org/10.1007/978-3-031-24670-8_57
Obo TTakizawa K(2022)Analysis of Timing and Effect of Visual Cue on Turn-Taking in Human-Robot InteractionJournal of Robotics and Mechatronics10.20965/jrm.2022.p005534:2(55-63)Online publication date: 20-Feb-2022
https://doi.org/10.20965/jrm.2022.p0055
Norton AAdmoni HCrandall JFitzgerald TGautam AGoodrich MSaretsky AScheutz MSimmons RSteinfeld AYanco H(2022)Metrics for Robot Proficiency Self-assessment and Communication of Proficiency in Human-robot TeamsACM Transactions on Human-Robot Interaction10.1145/352257911:3(1-38)Online publication date: 13-Jul-2022
https://dl.acm.org/doi/10.1145/3522579
Tang BSullivan DCagiltay BChandrasekaran VFawaz KMutlu B(2022)Confidant: A Privacy Controller for Social Robots2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889540(205-214)Online publication date: 7-Mar-2022
https://doi.org/10.1109/HRI53351.2022.9889540
Zhang ZDuan FCaiafa CSolé-Casals JYang ZSun Z(2022)Domain classifier-based transfer learning for visual attention predictionWorld Wide Web10.1007/s11280-022-01027-025:4(1685-1701)Online publication date: 27-Apr-2022
https://doi.org/10.1007/s11280-022-01027-0
Oliveira RArriaga PPaiva A(2021)Human-Robot Interaction in Groups: Methodological and Research PracticesMultimodal Technologies and Interaction10.3390/mti51000595:10(59)Online publication date: 30-Sep-2021
https://doi.org/10.3390/mti5100059
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents