research-article

What do you want to do next: a novel approach for intent prediction in gaze-based interaction

Authors:

Roman Bednarik,

Michal HradisAuthors Info & Claims

ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

Pages 83 - 90

https://doi.org/10.1145/2168556.2168569

Published: 28 March 2012 Publication History

Abstract

Interaction intent prediction and the Midas touch have been a longstanding challenge for eye-tracking researchers and users of gaze-based interaction. Inspired by machine learning approaches in biometric person authentication, we developed and tested an offline framework for task-independent prediction of interaction intents. We describe the principles of the method, the features extracted, normalization methods, and evaluation metrics. We systematically evaluated the proposed approach on an example dataset of gaze-augmented problem-solving sessions. We present results of three normalization methods, different feature sets and fusion of multiple feature types. Our results show that accuracy of up to 76% can be achieved with Area Under Curve around 80%. We discuss the possibility of applying the results for an online system capable of interaction intent prediction.

References

[1]

Ajanki, A., Hardoon, D. R., Kaski, S., Puolamäki, K., and Shawe-Taylor, J. 2009. Can eyes reveal interest? implicit queries from gaze patterns. User Model. User-Adapt. Interact. 19, 4, 307--339.

Digital Library

[2]

Bailey, B. P., and Iqbal, S. T. 2008. Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management. ACM Transactions on Computer-Human Interaction 14, 4 (Jan.), 1--28.

Digital Library

[3]

Beatty, J., and Lucero-Wagoner, B. 2000. The pupillary system. Cambridge University Press, ch. 6.

[4]

Beatty, J. 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychol Bull 91, 2 (Mar.), 276--292.

[5]

Bednarik, R., Kinnunen, T., Mihaila, A., and Fränti, P. 2005. Eye-movements as a biometric. In 14th Scandinavian Conference on Image Analysis, SCIA 2005, Springer, 780--789.

Digital Library

[6]

Bednarik, R., Gowases, T., and Tukiainen, M. 2009. Gaze interaction enhances problem solving: Effects of dwell-time based, gaze-augmented, and mouse interaction on problem-solving strategies and user experience. Journal of Eye Movement Research 3, 1, 3--10.

[7]

Bogert, B., Healy, M., and Tukey, J. 1963. The que-frency alanysis of time series for echoes: Cepstrum, Pseudo-Autocovariance, Cross-Cepstrum and Saphe Cracking. In Proc. Symp. on Time Series Analysis, 209--243.

[8]

Chang, C.-C., and Lin, C.-J. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1--27:27. Software available at http://www.csie.ntu.edu.tw/-cjlin/libsvm.

Digital Library

[9]

Cortes, C., and Vapnik, V. 1995. Support-vector networks. Machine Learning 20, 273--297. 10.1007/BF00994018.

[10]

Duggan, G. B., and Payne, S. J. 2011. Skim reading by satisficing: evidence from eye tracking. In Proceedings of the 2011 annual conference on Human factors in computing systems, ACM, New York, NY, USA, CHI '11, 1141--1150.

Digital Library

[11]

Egan, J. P. 1975. Signal Detection Theory and ROC Analysis. Academic Press.

[12]

Eivazi, S., and Bednarik, R. 2011. Predicting Problem-Solving Behavior and Performance Levels from Visual Attention Data. In 2nd Workshop on Eye Gaze in Intelligent Human Machine Interaction at ACM IUI 2011, 9--16.

[13]

Hupé, J.-M., Lamirel, C., and Lorenceau, J. 2009. Pupil dynamics during bistable motion perception. Journal of vision 9, 7 (Jan.), 10.

[14]

Iqbal, S. T., Zheng, X. S., and Bailey, B. P. 2004. Task-evoked pupillary response to mental workload in human-computer interaction. In CHI '04 extended abstracts on Human factors in computing systems, ACM, New York, NY, USA, CHI EA '04, 1477--1480.

Digital Library

[15]

Istance, H., Bates, R., Hyrskykari, A., and Vickers, S. 2008. Snap clutch, a moded approach to solving the midas touch problem. In Proceedings of the 2008 symposium on Eye tracking research & applications, ACM, New York, NY, USA, ETRA '08, 221--228.

Digital Library

[16]

Jacob, R. J. K., and Karn, K. S. 2003. Commentary on section 4. eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research. Elsevier Science, 573--605.

[17]

Jacob, R. J. K. 1991. The Use of Eye Movements in Interaction Techniques: What You Look At is What You Get. Human-Computer Interaction 9, 152--169.

Digital Library

[18]

Jones, E., Oliphant, T., Peterson, P., et al., 2001--. SciPy: Open source scientific tools for Python.

[19]

Kahneman, D. 1973. Attention and effort. Englewood Cliffs, Nj: Prentice-Hall.

[20]

Kandemir, M., Saarinen, V.-M., and Kaski, S. 2010. Inferring object relevance from gaze in dynamic scenes. In ETRA '10 Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, 105--108.

Digital Library

[21]

Kasprowski, P., and Ober, J. 2004. Eye movements in biometrics. In Biometric Authentication, D. Maltoni and A. Jain, Eds., vol. 3087 of Lecture Notes in Computer Science. Springer Berlin/Heidelberg, 248--258.

[22]

Kinnunen, T., Sedlak, F., and Bednarik, R. 2010. Towards task-independent person authentication using eye movement signals. In ETRA '10 Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, 187--190.

Digital Library

[23]

Klingner, J. 2010. Fixation-aligned pupillary response averaging. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA '10, ACM Press, New York, New York, USA, vol. 1, 275.

Digital Library

[24]

Majaranta, P., and Räihä, K.-J. 2002. Twenty years of eye typing: systems and design issues. In Proceedings of the 2002 symposium on Eye tracking research & applications, ACM, New York, NY, USA, ETRA '02, 15--22.

Digital Library

[25]

Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., and Euler, T. 2006. Yale: Rapid prototyping for complex data mining tasks. In KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, NY, USA, 935--940.

Digital Library

[26]

Oppenheim, A. V., Schafer, R. W., and Buck, J. R. 1999. Discrete-time signal processing (2nd ed.). Prentice-Hall, Inc., Upper Saddle River, NJ, USA.

Digital Library

[27]

Provost, F. J., Fawcett, T., and Kohavi, R. 1998. The case against accuracy estimation for comparing induction algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ICML '98, 445--453.

Digital Library

[28]

Richer, F., and Beatty, J. 1985. Pupillary dilations in movement preparation and execution. Psychophysiology 22, 2, 204--207.

[29]

Sakoe, H., and Chiba, S. 1990. Readings in speech recognition. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ch. Dynamic programming algorithm optimization for spoken word recognition, 159--165.

Digital Library

[30]

Salojärvi, J., Puolamäki, K., Simola, J., Kovanen, L., Kojo, I., and Kaski, S. 2005. Inferring relevance from eye movements: Feature extraction. In Helsinki University of Technology, No, 2005.

[31]

Simola, J., Salojärvi, J., and Kojo, I. 2008. Using hidden markov model to uncover processing states from eye movements in information search tasks. Cognitive Systems Research 9, 4, 237--251.

Digital Library

[32]

Tobii Technology AB, 2008. Clearview 2.7 eye gaze analysis software. http://www.tobii.com/scientific_research/.

Cited By

Barz MBednarik RBulling AConati CSonntag D(2024)HumanEYEze 2024: Workshop on Eye Tracking for Multimodal Human-Centric ComputingProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3688384(696-697)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3688384
Narkar AMichalak JPeacock CDavid-John B(2024)GazeIntent: Adapting Dwell-time Selection in VR Interaction with Real-time Intent ModelingProceedings of the ACM on Human-Computer Interaction10.1145/36556008:ETRA(1-18)Online publication date: 28-May-2024
https://dl.acm.org/doi/10.1145/3655600
Reddy GProulx MHirshfield LRies A(2024)Towards an Eye-Brain-Computer Interface: Combining Gaze with the Stimulus-Preceding Negativity for Target Selections in XRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641925(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641925
Show More Cited By

Index Terms

What do you want to do next: a novel approach for intent prediction in gaze-based interaction
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

EyeTAP: Introducing a multimodal gaze-based technique using voice inputs with a comparative analysis of selection techniques
Highlights
- Development of a new gaze-based interaction method: EyeTAP (Eye tracking point-and-select by Targeted Acoustic Pulse).
Abstract
One of the main challenges of gaze-based interactions is the ability to distinguish normal eye function from a deliberate interaction with the computer system, commonly referred to as ‘Midas touch’. In this paper we propose EyeTAP (Eye ...
GAWSCHI: gaze-augmented, wearable-supplemented computer-human interaction
ETRA '16: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications

Recent developments in eye tracking technology are paving the way for gaze-driven interaction as the primary interaction modality. Despite successful efforts, existing solutions to the "Midas Touch" problem have two inherent issues: 1) lower accuracy, ...
Designing for the eye: design parameters for dwell in gaze interaction
OzCHI '12: Proceedings of the 24th Australian Computer-Human Interaction Conference

Eye gaze tracking provides a natural and fast method of interacting with computers. Many click alternatives have been proposed so far, each with their own merits and drawbacks. We focus on the most natural selection method, i.e. the dwell, with which a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

March 2012

420 pages

ISBN:9781450312219

DOI:10.1145/2168556

Conference Chairs:
Carlos H. Morimoto
University of São Paulo, Brazil
,
Howell Istance
Montfort University, UK
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Jeffrey B. Mulligan
NASA
,
Pernilla Qvarfordt
FX Palo Alto

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 March 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ETRA '12

Sponsor:

ETRA '12: Eye Tracking Research and Applications

March 28 - 30, 2012

California, Santa Barbara

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

103
Total Citations
View Citations
1,976
Total Downloads

Downloads (Last 12 months)135
Downloads (Last 6 weeks)8

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Barz MBednarik RBulling AConati CSonntag D(2024)HumanEYEze 2024: Workshop on Eye Tracking for Multimodal Human-Centric ComputingProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3688384(696-697)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3688384
Narkar AMichalak JPeacock CDavid-John B(2024)GazeIntent: Adapting Dwell-time Selection in VR Interaction with Real-time Intent ModelingProceedings of the ACM on Human-Computer Interaction10.1145/36556008:ETRA(1-18)Online publication date: 28-May-2024
https://dl.acm.org/doi/10.1145/3655600
Reddy GProulx MHirshfield LRies A(2024)Towards an Eye-Brain-Computer Interface: Combining Gaze with the Stimulus-Preceding Negativity for Target Selections in XRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641925(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641925
Nishiyasu TSato Y(2024)Gaze Scanpath Transformer: Predicting Visual Search Target by Spatiotemporal Semantic Modeling of Gaze Scanpath2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00067(625-635)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00067
Kwok TKiefer PSchinazi VHoelscher CRaubal M(2024)Gaze-based detection of mind wandering during audio-guided panorama viewingScientific Reports10.1038/s41598-024-79172-x14:1Online publication date: 14-Nov-2024
https://doi.org/10.1038/s41598-024-79172-x
Yu MBi C(2024)Adaptive 360° video timeline exploration in VR environmentComputers & Graphics10.1016/j.cag.2024.104108125(104108)Online publication date: Dec-2024
https://doi.org/10.1016/j.cag.2024.104108
Zhang CLiao HHuang YDong W(2023)Evaluating the Usability of a Gaze-Adaptive Approach for Identifying and Comparing Raster Values between MultilayersISPRS International Journal of Geo-Information10.3390/ijgi1210041212:10(412)Online publication date: 8-Oct-2023
https://doi.org/10.3390/ijgi12100412
Dongye XWeng DJiang HFeng L(2023)A Modular Haptic Agent System with Encountered-Type Active InteractionElectronics10.3390/electronics1209206912:9(2069)Online publication date: 30-Apr-2023
https://doi.org/10.3390/electronics12092069
Isomoto TYamanaka SShizuki B(2023)Reanalyzing Effective Eye-related Information for Developing User’s Intent Detection SystemsProceedings of the 2023 Symposium on Eye Tracking Research and Applications10.1145/3588015.3589508(1-3)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588015.3589508
Chan SZhang HNanayakkara S(2023)Eye Movement Analysis of Human Visual Recognition Processes with Camera Eye Tracker: Higher Mean and Variance of Fixation Duration for Familiar ImagesExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585782(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585782
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten