research-article

Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction

Authors:

Carlos T. Ishi,

Hiroshi Ishiguro,

Norihiro HagitaAuthors Info & Claims

HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

Pages 285 - 292

https://doi.org/10.1145/2157689.2157797

Published: 05 March 2012 Publication History

Abstract

Head motion occurs naturally and in synchrony with speech during human dialogue communication, and may carry paralinguistic information, such as intentions, attitudes and emotions. Therefore, natural-looking head motion by a robot is important for smooth human-robot interaction. Based on rules inferred from analyses of the relationship between head motion and dialogue acts, this paper proposes a model for generating head tilting and nodding, and evaluates the model using three types of humanoid robot (a very human-like android, "Geminoid F", a typical humanoid robot with less facial degrees of freedom, "Robovie R2", and a robot with a 3-axis rotatable neck and movable lips, "Telenoid R2"). Analysis of subjective scores shows that the proposed model including head tilting and nodding can generate head motion with increased naturalness compared to nodding only or directly mapping people's original motions without gaze information. We also find that an upwards motion of a robot's face can be used by robots which do not have a mouth in order to provide the appearance that utterance is taking place. Finally, we conduct an experiment in which participants act as visitors to an information desk attended by robots. As a consequence, we verify that our generation model performs equally to directly mapping people's original motions with gaze information in terms of perceived naturalness.

Supplementary Material

JPG File (hri200.jpg)

Download
8.96 KB

AVI File (hri200.avi)

Download
23.50 MB

References

[1]

C. T. Ishi, C. Liu, H. Ishiguro, N. Hagita. 2010. Head motion during dialogue speech and nod timing control in humanoid robots. Proc. of IEEE/RSJ Human Robot Interaction (HRI 2010), 293--300, 2010.

Digital Library

[2]

C. Sidner, C. Lee, L.-P. Morency, C. Forlines. 2006. The effect of head-nod recognition in human-robot conversation. Proc. of IEEE/RSJ Human Robot Interaction (HRI 2006), pp. 290--296, 2006.

Digital Library

[3]

L.-P. Morency, C. Sidner, C. Lee, and T. Darrell. 2007. Head gestures for perceptual interfaces: The role of context in improving recognition. Artificial Intelligence, 171(8-9): 568--585, June 2007.

Digital Library

[4]

H. C. Yehia, T. Kuratate, E. Vatikiotis-Bateson. 2002. Linking facial animation, head motion and speech acoustics. J. of Phonetics, Vol. 30, pp. 555--568, 2002.

[5]

M. E. Sargin, O. Aran, A. Karpov, F. Ofli, Y. Yasinnik, S. Wilson, E. Erzin, Y. Yemez, A. M. Tekalp. 2006. Combined Gesture-Speech Analysis and Speech Driven Gesture Synthesis. Proc IEEE International Conference on Multimedia, 2006.

[6]

K. G. Munhall, J. A. Jones, D. E. Callan, T. Kuratate, E. Vatikiotis-Bateson. 2004. Visual prosody and speech intelligibility - Head movement improves auditory speech perception. Psychological Science, Vol. 15, No. 2, pp. 133--137, 2004.

[7]

H. P. Graf., E. Cosatto, V. Strom, F. J. Huang. 2002. Visual prosody: Facial movements accompanying speech. Proc. IEEE Int. Conf. on Automatic Face and Gesture Recognition (FGR'02), 2002.

Digital Library

[8]

J. Beskow, B. Granstrom, D. House. 2006. Visual correlates to prominence in several expressive modes. Proc. Interspeech 2006 - ICSLP, pp. 1272--1275, 2006.

[9]

C. Busso, Z. Deng, M. Grimm, U. Neumann, S. Narayanan. 2007. Rigid Head Motion in Expressive Speech Animation: Analysis and Synthesis. IEEE Trans. on Audio, Speech and Language Processing, March 2007.

Digital Library

[10]

Y. Iwano, S. Kageyama, E. Morikawa, S. Nakazato, K. Shirai. 1996. Analysis of head movements and its role in spoken dialogue. Proc. International Conference on Spoken Language Processing (ICSLP'96), pp. 2167--2170, 1996.

[11]

M. E. Foster and J. Oberlander. 2007. Corpus-based generation of head and eyebrow motion for an embodied conversational agent. Language Resources and Evaluation, 41(3-4), 305--323, Dec. 2007.

[12]

C. T Ishi, J. Haas, F. P. Wilbers, H. Ishiguro, and N. Hagita. 2007. Analysis of head motions and speech, and head motion control in an android. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), 548--553, 2007.

[13]

C. T. Ishi, H. Ishiguro, N. Hagita. 2006. Analysis of prosodic and linguistic cues of phrase finals for turn-taking and dialog acts. Proc. Interspeech'2006 - ICSLP, pp. 2006--2009, 2006.

[14]

T. Minato, M. Shimada, H. Ishiguro, S. Itakura. 2004. Development of an android robot for studying human-robot interaction. Innovations in Applied Artificial Intelligence, Springer Verlag, pp. 424--434, 2004.

Digital Library

[15]

M. DeBoer and A. M. Boxer. 1979. Signal functions of infant facial expression and gaze direction during mother-infant face-to-face play. Child Development, vol. 50, no. 4, pp. 1215--1218, 1979.

[16]

S. R. H. Langton, R. J. Watt, and V. Bruce. 2000. Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences, vol. 4, no. 2, pp. 50--59, 2000.

[17]

F. Kaplan and V. Hafner. 2004. The challenges of joint attention. Interaction Studies, pp. 67--74, 2004. {Online}. Available: http://cogprints.org/4067/

[18]

Y. Nagai, M. Asada, and K. Hosoda. 2006. Learning for joint attention helped by functional development. Advanced Robotics, vol. 20, pp. 1165--1181(17), October 2006.

[19]

C. T. Ishi, C. Liu, H. Ishiguro, N. Hagita. 2011. Speech-driven lip motion generation for tele-operated humanoid robots. International Conference on Auditory-Visual Speech Processing, 2011

Cited By

Koike AWehner MMutlu BGrollman DBroadbent EJu WSoh HWilliams T(2024)Sprout: Designing Expressivity for Robots Using Fiber-Embedded ActuatorProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634983(403-412)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634983
Maehigashi ATsumura TYamada S(2024)Impacts of Robot Beep Timings on Trust Dynamics in Human-Robot InteractionInternational Journal of Social Robotics10.1007/s12369-024-01181-7Online publication date: 24-Oct-2024
https://doi.org/10.1007/s12369-024-01181-7
Noguchi SNakamura YOkadome Y(2024)Development of a Attentive Listening Robot Using the Motion Prediction Based on Surrogate DataHCI International 2024 Posters10.1007/978-3-031-61932-8_44(387-394)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61932-8_44
Show More Cited By

Index Terms

Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
      1. External interfaces for robotics

Recommendations

Precision timing in human-robot interaction: coordination of head movement and utterance
CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

As research over the last several decades has shown that non-verbal actions such as face and head movement play a crucial role in human interaction, such resources are also likely to play an important role in human-robot interaction. In developing a ...
Head motions during dialogue speech and nod timing control in humanoid robots
HRI '10: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction

Head motion naturally occurs in synchrony with speech and may carry paralinguistic information, such as intention, attitude and emotion, in dialogue communication. With the aim of verifying the relationship between head motion and the dialogue acts ...
Damping robot's head movements affects human-robot interaction
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction

A new research platform has been developed to study human-robot interaction and communication. In this setup, a humanoid robot is used as a proxy between two humans involved in dyadic interactions. An experimenter is bound with a humanoid robot. He can ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

March 2012

518 pages

ISBN:9781450310635

DOI:10.1145/2157689

General Chairs:
Holly Yanco
University of Massachusetts Lowell, USA
,
Aaron Steinfeld
Carnegie Mellon University, USA
,
Program Chairs:
Vanessa Evers
University of Amsterdam, The Netherlands
,
Odest Chadwicke Jenkins
Brown University, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HRI'12

Sponsor:

HRI'12: International Conference on Human-Robot Interaction

March 5 - 8, 2012

Massachusetts, Boston, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
638
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)18

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Koike AWehner MMutlu BGrollman DBroadbent EJu WSoh HWilliams T(2024)Sprout: Designing Expressivity for Robots Using Fiber-Embedded ActuatorProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634983(403-412)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634983
Maehigashi ATsumura TYamada S(2024)Impacts of Robot Beep Timings on Trust Dynamics in Human-Robot InteractionInternational Journal of Social Robotics10.1007/s12369-024-01181-7Online publication date: 24-Oct-2024
https://doi.org/10.1007/s12369-024-01181-7
Noguchi SNakamura YOkadome Y(2024)Development of a Attentive Listening Robot Using the Motion Prediction Based on Surrogate DataHCI International 2024 Posters10.1007/978-3-031-61932-8_44(387-394)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61932-8_44
YOKOTA MHISATOMI AISHII YWATANABE T(2023)A voice-driven embodied interaction system with response actions according to speech speed by the moraモーラに基づく発話速度に応じた反応動作を行う音声駆動型身体引き込みシステムTransactions of the JSME (in Japanese)10.1299/transjsme.22-0022889:919(22-00228-22-00228)Online publication date: 2023
https://doi.org/10.1299/transjsme.22-00228
Chang FZeng JLiu QShan S(2023)Gaze Pattern Recognition in Dyadic CommunicationProceedings of the 2023 Symposium on Eye Tracking Research and Applications10.1145/3588015.3588411(1-7)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588015.3588411
Pelikan HHofstetter E(2023)Managing Delays in Human-Robot InteractionACM Transactions on Computer-Human Interaction10.1145/356989030:4(1-42)Online publication date: 12-Sep-2023
https://dl.acm.org/doi/10.1145/3569890
Moon CYamsani SKim J(2023)Development of a 3-DOF Interactive Modular Robot with Human-like Head Motions2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309462(141-146)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309462
Seaborn KBarbareschi GChandra S(2023)Not Only WEIRD but “Uncanny”? A Systematic Review of Diversity in Human–Robot Interaction ResearchInternational Journal of Social Robotics10.1007/s12369-023-00968-415:11(1841-1870)Online publication date: 8-Mar-2023
https://doi.org/10.1007/s12369-023-00968-4
Colley ATiitto MPfleging BHäkkilä J(2023)Pedestrian Interaction with a Snow Clearing RobotHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42283-6_1(3-16)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42283-6_1
Marina-Miranda JTraver V(2022)Head and Eye Egocentric Gesture Recognition for Human-Robot Interaction Using Eyewear CamerasIEEE Robotics and Automation Letters10.1109/LRA.2022.31804427:3(7067-7074)Online publication date: Jul-2022
https://doi.org/10.1109/LRA.2022.3180442
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents