research-article

On the design and evaluation of robust head pose for visual user interfaces: algorithms, databases, and comparisons

Authors:

Sujitha Martin,

Erik Murphy-Chutorian,

Shinko Y. Cheng,

Mohan TrivediAuthors Info & Claims

AutomotiveUI '12: Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Pages 149 - 154

https://doi.org/10.1145/2390256.2390281

Published: 17 October 2012 Publication History

Abstract

An important goal in automotive user interface research is to predict a user's reactions and behaviors in a driving environment. The behavior of both drivers and passengers can be studied by analyzing eye gaze, head, hand, and foot movement, upper body posture, etc. In this paper, we focus on estimating head pose, which has been shown to be a good predictor of driver intent and a good proxy for gaze estimation, and provide a valuable head pose database for future comparative studies. Most existing head pose estimation algorithms are still struggling under large spatial head turns. Our method, however, relies on using facial features that are visible even during large spatial head turns to estimate head pose. The method is evaluated on the LISA-P Head Pose database, which has head pose data from on-road daytime and nighttime drivers of varying age, race, and gender; ground truth for head pose is provided using a motion capture system. In special regards to eye gaze estimation for automotive user interface study, the automatic head pose estimation technique presented in this paper can replace previous eye gaze estimation methods that rely on manual data annotation or be used in conjunction with them when necessary.

References

[1]

S. Ba and J.-M. Odobez. Evaluation of multiple cue head pose estimation algorithms in natural environements. In Multimedia and Expo, 2005. ICME 2005. IEEE International Conference on, pages 1330--1333, july 2005.

[2]

J. Busby. 3d head scan, August 2012.

[3]

J. Chen and Q. Ji. 3d gaze estimation with a single camera without ir illumination. In Pattern Recognition, 2008. ICPR 2008. 19th International Conference on, pages 1--4, dec. 2008.

[4]

S. Cheng and M. Trivedi. Turn-intent analysis using body pose for intelligent driver assistance. Pervasive Computing, IEEE, 5(4):28--37, oct.-dec. 2006.

Digital Library

[5]

L. H. Christiansen, N. Y. Frederiksen, A. Ranch, and M. B. Skov. Investigating the effects of an advance warning in-vehicle system on behavior and attention in controlled driving. In Proceedings of the 3rd Internation Conference on Automotive User Interface and Interactive Vehicular Applications, pages 121--128, 2011.

Digital Library

[6]

J. Curin, M. Labsky, T. Macek, and J. Kleindienst. Dictating and editing short texts while driving: Distraction and task completion. In Proceedings of the 3rd Internation Conference on Automotive User Interface and Interactive Vehicular Applications, AutomotiveUI '11, pages 13--20, 2011.

Digital Library

[7]

D. F. Dementhon and L. S. Davis. Model-based object pose in 25 lines of code. International Journal of Computer Vision, 15:123--141, 1995.

Digital Library

[8]

A. Doshi and M. Trivedi. On the roles of eye gaze and head dynamics in predicting driver's intent to change lanes. Intelligent Transportation Systems, IEEE Transactions on, 10(3):453--462, sept. 2009.

Digital Library

[9]

L. Fletcher, L. Petersson, N. Barnes, D. Austin, and A. Zelinsky. A sign reading driver assistance system using eye gaze. In Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International Conference on, pages 4655--4660, april 2005.

[10]

L. Fletcher, L. Petersson, and A. Zelinsky. Road scene monotony detection in a fatigue management driver assistance system. In Intelligent Vehicles Symposium, 2005. Proceedings. IEEE, pages 484--489, june 2005.

[11]

A. Gee and R. Cipolla. Determining the gaze of faces in images. Image and Vision Computing, 12(10):639--647, 1994.

[12]

D. Hansen and Q. Ji. In the eye of the beholder: A survey of models for eyes and gaze. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(3):478--500, march 2010.

Digital Library

[13]

T. Horprasert, Y. Yacoob, and L. Davis. Computing 3-d head orientation from a monocular image sequence. In Automatic Face and Gesture Recognition, 1996., Proceedings of the Second International Conference on, pages 242--247, oct 1996.

Digital Library

[14]

T. Ishikawa, S. Baker, I. Matthews, and T. Kanade. Passive driver gaze tracking with active appearance models. In Proceedings of the 11th World Congress on Intelligent Transportation Systems, October 2004.

[15]

J. Jain and A. Jain. Displacement measurement and its application in interframe image coding. Communications, IEEE Transactions on, 29(12):1799--1808, dec 1981.

[16]

M. La Cascia and S. Sclaroff. Fast, reliable head tracking under varying illumination. In Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on., volume 1, pages 2 vol. (xxiii+637+663), 1999.

[17]

S. Martin, C. Tran, A. Tawari, J. Kwan, and M. M. Trivedi. Optical flow based head movement and gesture analyzer (ohmega). In Pattern Recognition (ICPR), 21st International Conference on, Nov. 2012.

[18]

S. Martin, C. Tran, and M. M. Trivedi. Optical flow based head movement and gesture analysis in automotive environment. In IEEE International Conference on Intelligent Transportation Systems-ITSC, Sept. 2012.

[19]

J. McCall, D. Wipf, M. Trivedi, and B. Rao. Lane change intent analysis using robust operators and sparse bayesian learning. Intelligent Transportation Systems, IEEE Transactions on, 8(3):431--440, sept. 2007.

Digital Library

[20]

E. Murphy-Chutorian and M. Trivedi. Hyhope: Hybrid head orientation and position estimation for vision-based driver head tracking. In Intelligent Vehicles Symposium, 2008 IEEE, pages 512--517, june 2008.

[21]

E. Murphy-Chutorian and M. Trivedi. Head pose estimation in computer vision: A survey. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(4):607--626, april 2009.

Digital Library

[22]

E. Murphy-Chutorian and M. Trivedi. Head pose estimation and augmented reality tracking: An integrated system and evaluation for monitoring driver awareness. Intelligent Transportation Systems, IEEE Transactions on, 11(2):300--311, june 2010.

Digital Library

[23]

J. Saragih, S. Lucey, and J. Cohn. Face alignment through subspace constrained mean-shifts. In Computer Vision, 2009 IEEE 12th International Conference on, pages 1034--1041, 29 2009-oct. 2 2009.

[24]

T. Sim, S. Baker, and M. Bsat. The cmu pose, illumination, and expression (pie) database. In Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on, pages 46--51, may 2002.

Digital Library

[25]

G. Slabaugh. Computing euler angles from a rotation matrix.

[26]

R. Valenti, N. Sebe, and T. Gevers. Combining head pose and eye location information for gaze estimation. Image Processing, IEEE Transactions on, 21(2):802--815, feb. 2012.

Digital Library

[27]

J.-G. Wang and E. Sung. Em enhancement of 3d head pose estimated by point at infinity. Image and Vision Computing, 25(12):1864--1874, 2007. The age of human computer interaction.

Digital Library

[28]

J. Wu and M. M. Trivedi. A two-stage head pose estimation framework and evaluation. Pattern Recognition, 41(3):1138--1158, 2008. Part Special issue: Feature Generation and Machine Learning for Robust Multimodal Biometrics.

Digital Library

[29]

H. Zhang, M. Smith, and R. Dufour. A final report of safety vehicles using adaptive interface technology: Visual distraction.

Cited By

Gao FGe XLi JFan YLi YZhao R(2024)Intelligent Cockpits for Connected Vehicles: Taxonomy, Architecture, Interaction Technologies, and Future DirectionsSensors10.3390/s2416517224:16(5172)Online publication date: 10-Aug-2024
https://doi.org/10.3390/s24165172
Hu JJiang HLiu DXiao ZZhang QLiu JDustdar S(2024)Combining IMU With Acoustics for Head Motion Tracking Leveraging Wireless EarphoneIEEE Transactions on Mobile Computing10.1109/TMC.2023.332582623:6(6835-6847)Online publication date: Jun-2024
https://doi.org/10.1109/TMC.2023.3325826
Wang JLi WLi FZhang JWu ZZhong ZSebe N(2023)100-Driver: A Large-Scale, Diverse Dataset for Distracted Driver ClassificationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.325592324:7(7061-7072)Online publication date: Jul-2023
https://doi.org/10.1109/TITS.2023.3255923
Show More Cited By

Index Terms

On the design and evaluation of robust head pose for visual user interfaces: algorithms, databases, and comparisons
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Interest point and salient region detections

Recommendations

Continuous Emotion Recognition in Videos by Fusing Facial Expression, Head Pose and Eye Gaze
ICMI '19: 2019 International Conference on Multimodal Interaction

Continuous emotion recognition is of great significance in affective computing and human-computer interaction. Most of existing methods for video based continuous emotion recognition utilize facial expression. However, besides facial expression, other ...
Head orientation and gaze direction in meetings
CHI EA '02: CHI '02 Extended Abstracts on Human Factors in Computing Systems

Detecting who is looking at whom during multiparty interaction is useful for various tasks such as meeting analysis. There are two contributing factors in the formation of where a person is looking at : head orientation and eye orientation. In this ...
Gaze data collection with the off-the-shelf devices
PCM'10: Proceedings of the Advances in multimedia information processing, and 11th Pacific Rim conference on Multimedia: Part II

Gaze is a very important modality in Human-computer Interaction (HCI). Sufficient gaze data is one of the vital foundations for automatic gaze tracking. In this paper, we propose a method to collect automatic labelled gaze data with the off-the-shelf ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AutomotiveUI '12: Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

October 2012

280 pages

ISBN:9781450317511

DOI:10.1145/2390256

General Chair:
Andrew L. Kun
University of New Hampshire
,
Program Chairs:
Linda Boyle
University of Washington
,
Bryan Reimer
Massachusetts Institute of Technology
,
Andreas Riener
University of Linz
,
Jennifer Healey
Intel Corporation
,
Wei Zhang
Tsinghua University
,
Publications Chairs:
Bastian Pfleging
University of Stuttgart
,
Marc Kurz
University of Linz

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AutomotiveUI '12

AutomotiveUI '12: International Conference on Automotive User Interfaces and Interactive Vehicular Applications

October 17 - 19, 2012

New Hampshire, Portsmouth

Acceptance Rates

Overall Acceptance Rate 248 of 566 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
458
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gao FGe XLi JFan YLi YZhao R(2024)Intelligent Cockpits for Connected Vehicles: Taxonomy, Architecture, Interaction Technologies, and Future DirectionsSensors10.3390/s2416517224:16(5172)Online publication date: 10-Aug-2024
https://doi.org/10.3390/s24165172
Hu JJiang HLiu DXiao ZZhang QLiu JDustdar S(2024)Combining IMU With Acoustics for Head Motion Tracking Leveraging Wireless EarphoneIEEE Transactions on Mobile Computing10.1109/TMC.2023.332582623:6(6835-6847)Online publication date: Jun-2024
https://doi.org/10.1109/TMC.2023.3325826
Wang JLi WLi FZhang JWu ZZhong ZSebe N(2023)100-Driver: A Large-Scale, Diverse Dataset for Distracted Driver ClassificationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.325592324:7(7061-7072)Online publication date: Jul-2023
https://doi.org/10.1109/TITS.2023.3255923
Stampf AColley MRukzio E(2022)Towards Implicit Interaction in Highly Automated Vehicles - A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/35467266:MHCI(1-21)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1145/3546726
Zhou YChen HHuang CZhang Q(2022)WiAdvProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346186:2(1-25)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534618
Jansen PColley MRukzio E(2022)A Design Space for Human Sensor and Actuator Focused In-Vehicle Interaction Based on a Systematic Literature ReviewProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346176:2(1-51)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534617
Parilusyan BTeyssier MMartinez-Missir VDuhart CSerrano M(2022)SensurfacesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346166:2(1-19)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534616
Gupta KChan SPai YStrachan NSu JSumich ANanayakkara SBillinghurst M(2022)Total VREcallProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346156:2(1-21)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3534615
Jha SMarzban MHu TMahmoud MAl-Dhahir NBusso C(2022)The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver AttentionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2021.309546223:8(10736-10752)Online publication date: Aug-2022
https://doi.org/10.1109/TITS.2021.3095462
Adachi JTsukahara HMizuno NYoshizawa A(2022)Action Inference of Rear Seat Passenger for In-Vehicle Service2022 IEEE Intelligent Vehicles Symposium (IV)10.1109/IV51971.2022.9827225(77-82)Online publication date: 5-Jun-2022
https://doi.org/10.1109/IV51971.2022.9827225
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten