Effect of Obstacle Type and Cognitive Task on Situation Awareness and Takeover Performance in Conditionally Automated Driving
Authors: 
Quentin Meteier, Marine Capallera, Emmanuel De Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, Elena Mugellini, Andreas SondereggerAuthors Info & Claims
Quentin Meteier, Marine Capallera, Emmanuel De Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, Elena Mugellini, Andreas SondereggerAuthors Info & ClaimsArticle No.: 5, Pages 1 - 12
Abstract
In conditionally automated driving, several factors can affect the driver’s situation awareness and ability to take over control. To better understand the influence of some of these factors, 88 participants spent 20 minutes in a conditionally automated driving simulator. They had to react to four obstacles that varied in danger and movement. Half of the participants were required to engage in a verbal cognitive non-driving-related task. Situation awareness, takeover performance and physiological responses were measured for each situation. The results suggest that obstacle movement influences obstacle danger perception, situation awareness, and response time, while the latter is also influenced by obstacle danger. The cognitive verbal task also had an effect on the takeover response time. These results imply that the driver’s cognitive state and the driving situation (e.g. the movement/danger of entities present around the vehicle) must be considered when conveying information to drivers through in-vehicle interfaces.
References
[1]
Shubham Agrawal and Srinivas Peeta. 2021. Evaluating the impacts of situational awareness and mental stress on takeover performance under conditional automation. Transportation research part F: traffic psychology and behaviour 83 (2021), 210–225.
[2]
Pavlo Bazilinskyy and Joost de Winter. 2018. Crowdsourced Measurement of Reaction Times to Audiovisual Stimuli With Various Degrees of Asynchrony. Human factors 60, 8 (Dec 2018), 1192–1206. https://doi.org/10.1177/0018720818787126
[3]
Wolfram Boucsein. 2012. Electrodermal Activity (2 ed.). Springer US.
[4]
Mercedes Bueno, Ebru Dogan, Fouad Hadj-Selem, Éric Monacelli, Serge Boverie, and Anne Guillaume. 2016. How different mental workload levels affect the take-over control after automated driving. 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC) (2016), 2040–2045.
[5]
Marine Capallera, Peïo Barbé-Labarthe, Leonardo Angelini, Omar Abou Khaled, and Elena Mugellini. 2019. Convey situation awareness in conditionally automated driving with a haptic seat. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings (2019).
[6]
Marine Capallera, Emmanuel de Salis, Quentin Meteier, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, and Elena Mugellini. 2019. Secondary task and situation awareness, a mobile application for conditionally automated vehicles. Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings (2019).
[7]
Marine Capallera, Quentin Meteier, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, and Elena Mugellini. 2019. Owner Manuals Review and Taxonomy of ADAS Limitations in Partially Automated Vehicles. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 156–164. https://doi.org/10.1145/3342197.3344530
[8]
Theresa Chirles, Johnathon P. Ehsani, Neale Kinnear, and Karen E Seymour. 2021. Skin Conductance Responses of Learner and Licensed Drivers During a Hazard Perception Task. Frontiers in Psychology 12 (2021).
[9]
H. Cohen, M. Kotler, M. Matar, Z. Kaplan, and Y. Cassuto. 1998. Analysis of heart rate variability in posttraumatic stress disorder patients in response to a trauma-related reminder. Biological Psychiatry 44 (1998), 1054–1059.
[10]
C. Collet, A. Clarion, M. Morel, A. Chapon, and C. Petit. 2009. Physiological and behavioural changes associated to the management of secondary tasks while driving. Applied Ergonomics 40, 6 (Nov. 2009), 1041–1046. https://doi.org/10.1016/j.apergo.2009.01.007
[11]
Benjamin Cortens, Blair Nonnecke, and Lana M. Trick. 2019. Effect of Alert Presentation Mode and Hazard Direction on Driver Takeover from an Autonomous Vehicle. Proceedings of the 10th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design: driving assessment 2019 (2019).
[12]
B. Cuthbert, P. Lang, C. Strauss, D. Drobes, C. Patrick, and M. Bradley. 2003. The psychophysiology of anxiety disorder: fear memory imagery.Psychophysiology 40 3 (2003), 407–22.
[13]
Michael E. Dawson, Anne M. Schell, and Dianne L Filion. 2007. The Electrodermal System. In Handbook of psychophysiology, J.T. Cacioppo, L.G. Tassinary, and G. Berntson (Eds.). Cambridge University Press, Chapter 7, 159–181. https://doi.org/10.1017/CBO9780511546396.007
[14]
Emmanuel de Salis, Marine Capallera, Quentin Meteier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, and Stefano Carrino. 2020. Designing an AI-Companion to Support the Driver in Highly Autonomous Cars. In HCI.
[15]
Na Du, Jinyong Kim, Feng Zhou, Elizabeth Pulver, Dawn M. Tilbury, Lionel Peter Robert, Anuj K. Pradhan, and X. Jessie Yang. 2020. Evaluating Effects of Cognitive Load, Takeover Request Lead Time, and Traffic Density on Drivers’ Takeover Performance in Conditionally Automated Driving. 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (2020).
[16]
M. Endsley. 1988. Situation awareness global assessment technique (SAGAT). Proceedings of the IEEE 1988 National Aerospace and Electronics Conference (1988), 789–795 vol.3.
[17]
Mica R. Endsley. 1995. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37, 1 (Mar 1995), 32–64. https://doi.org/10.1518/001872095779049543
[18]
Alexandra Fernandes and Per Øivind Braarud. 2015. Exploring Measures of Workload, Situation Awareness, and Task Performance in the Main Control Room. Procedia Manufacturing 3 (2015), 1281–1288. https://doi.org/10.1016/j.promfg.2015.07.273 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015.
[19]
Kelly Funkhouser and Frank Drews. 2016. Reaction Times When Switching From Autonomous to Manual Driving Control: A Pilot Investigation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 60, 1 (2016), 1854–1858. https://doi.org/10.1177/1541931213601423 arXiv:https://doi.org/10.1177/1541931213601423
[20]
V. Greco and D. Roger. 2003. Uncertainty, stress, and health. Personality and Individual Differences 34 (2003), 1057–1068.
[21]
S. G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50 (2006), 904 – 908.
[22]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology, Peter A. Hancock and Najmedin Meshkati (Eds.). Human Mental Workload, Vol. 52. North-Holland, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9
[23]
SAE International. 2018. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles.
[24]
Oliver Jarosch, Hanna Bellem, and Klaus Bengler. 2019. Effects of Task-Induced Fatigue in Prolonged Conditional Automated Driving. Human Factors: The Journal of Human Factors and Ergonomics Society 61 (2019), 1186 – 1199.
[25]
David B. Kaber, Yu Zhang, Sangeun Jin, Prithima Mosaly, and Megan Garner. 2012. Effects of hazard exposure and roadway complexity on young and older driver situation awareness and performance. Transportation Research Part F-traffic Psychology and Behaviour 15 (2012), 600–611.
[26]
Roger Kirk. 2013. Latin square and related designs. In Experimental Design: Procedures for the Behavioral Sciences (4 ed.). SAGE Publications, Inc., Thousand Oaks, California. https://doi.org/10.4135/9781483384733
[27]
David R. Large, Gary Burnett, Davide Salanitri, Anneka Lawson, and Elizabeth Box. 2019. A Longitudinal Simulator Study to Explore Drivers’ Behaviour in Level 3 Automated Vehicles. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM, 222–232. https://doi.org/10.1145/3342197.3344519
[28]
Nengchao Lyu, Lian Xie, Chaozhong Wu, Qiang Fu, and Chao Deng. 2017. Driver’s Cognitive Workload and Driving Performance under Traffic Sign Information Exposure in Complex Environments: A Case Study of the Highways in China. International Journal of Environmental Research and Public Health 14 (2017).
[29]
Megan K. MacPherson, D. Abur, and C. Stepp. 2017. Acoustic Measures of Voice and Physiologic Measures of Autonomic Arousal during Speech as a Function of Cognitive Load.Journal of voice : official journal of the Voice Foundation 31 4 (2017), 504.e1–504.e9.
[30]
Dominique Makowski, Tam Pham, Zen J. Lau, Jan C. Brammer, François Lespinasse, Hung Pham, Christopher Schölzel, and S. H. Annabel Chen. 2021. NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior Research Methods (02 Feb 2021). https://doi.org/10.3758/s13428-020-01516-y
[31]
Bruce Mehler, B. Reimer, and J. Coughlin. 2012. Sensitivity of Physiological Measures for Detecting Systematic Variations in Cognitive Demand From a Working Memory Task. Human Factors: The Journal of Human Factors and Ergonomics Society 54 (2012), 396 – 412.
[32]
Bruce Mehler, Bryan Reimer, Joseph Coughlin, and Jeffery Dusek. 2009. The Impact of Incremental Increases in Cognitive Workload on Physiological Arousal and Performance in Young Adult Drivers. Transportation Research Record: Journal of the Transportation Research Board 2138 (Dec. 2009), 6–12. https://doi.org/10.3141/2138-02
[33]
Bruce Mehler, B. Reimer, and Y. Wang. 2017. A Comparison of Heart Rate and Heart Rate Variability Indices in Distinguishing Single-Task Driving and Driving Under Secondary Cognitive Workload.
[34]
Natasha Merat, A. Hamish Jamson, Frank C. H. Lai, and Oliver M. J. Carsten. 2012. Highly Automated Driving, Secondary Task Performance, and Driver State. Human Factors: The Journal of Human Factors and Ergonomics Society 54 (2012), 762 – 771.
[35]
Quentin Meteier, Marine Capallera, Emmanuel de Salis, Andreas Sonderegger, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, and Elena Mugellini. 2020. The Effect of Instructions and Context-Related Information about Limitations of Conditionally Automated Vehicles on Situation Awareness. 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (2020).
[36]
Quentin Meteier, Marine Capallera, Simon Ruffieux, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, and Andreas Sonderegger. 2021. Classification of Drivers’ Workload Using Physiological Signals in Conditional Automation. Frontiers in Psychology 12 (2021).
[37]
Quentin Meteier, Emmanuel de Salis, Marine Capallera, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Andreas Sonderegger, and Elena Mugellini. 2021. Relevant Physiological Indicators for Assessing Workload in Conditionally Automated Driving, Through Three-Class Classification and Regression. In Frontiers in Computer Science.
[38]
Brian Mok, Mishel Johns, David Miller, and Wendy Ju. 2017. Tunneled in: Drivers with active secondary tasks need more time to transition from automation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2840–2844.
[39]
Frederik Naujoks, Christian Purucker, Katharina Wiedemann, and Claus Marberger. 2019. Noncritical State Transitions During Conditionally Automated Driving on German Freeways: Effects of Non–Driving Related Tasks on Takeover Time and Takeover Quality. Human Factors: The Journal of Human Factors and Ergonomics Society 61 (2019), 596 – 613.
[40]
Thanh Nguyen, Chee Peng Lim, Ngoc Duy Nguyen, Lee Gordon-Brown, and Saeid Nahavandi. 2019. A Review of Situation Awareness Assessment Approaches in Aviation Environments. (2019), 16.
[41]
Julie Paxion, Edith Galy, and Catherine Berthelon. 2014. Mental workload and driving. Frontiers in Psychology 5 (2014).
[42]
Bryan Reimer and Bruce Mehler. 2013. The Effects of a Production Level “Voice-Command” Interface on Driver Behavior: Summary Findings on Reported Workload, Physiology, Visual Attention, and Driving Performance. (2013), 20.
[43]
Janette Rose, Chris Bearman, Anjum Naweed, and Jillian Dorrian. 2019. Proceed with caution: using verbal protocol analysis to measure situation awareness. Ergonomics 62, 1 (2019), 115–127. https://doi.org/10.1080/00140139.2018.1527951 arXiv:https://doi.org/10.1080/00140139.2018.1527951PMID: 30265217.
[44]
Marlene Susanne Lisa Scharfe, Kathrin Zeeb, and Nele Rußwinkel. 2020. The Impact of Situational Complexity and Familiarity on Takeover Quality in Uncritical Highly Automated Driving Scenarios. Inf. 11 (2020), 115.
[45]
F Shaffer, S Shearman, Z Meehan, N Gravett, and H Urban. 2019. The promise of ultra-short-term (UST) heart rate variability measurements: a comparison of Pearson product-moment correlation coefficient and limits of agreement (LoA) concurrent validity criteria. Physiological Recording Technology and Applications in Biofeedback and Neurofeedback, eds D. Moss and F. Shaffer (Oakbrook Terrace, IL: Association for Applied Psychophysiology and Biofeedback) (2019), 214–220.
[46]
Yoshihiro Shimomura, Takumi Yoda, Koji Sugiura, Akinori Horiguchi, Koichi Iwanaga, and Tetsuo Katsuura. 2008. Use of frequency domain analysis of skin conductance for evaluation of mental workload.Journal of physiological anthropology 27 4 (2008), 173–7.
[47]
Erin Treacy Solovey, Marin Zec, Enrique Abdon Garcia Perez, B. Reimer, and Bruce Mehler. 2014. Classifying driver workload using physiological and driving performance data: two field studies. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2014).
[48]
Richard M Taylor. 2017. Situational awareness rating technique (SART): The development of a tool for aircrew systems design. In Situational awareness. Routledge, 111–128.
[49]
John R Treat, Nicholas S Tumbas, Stephen T McDonald, David Shinar, Rex D Hume, RE Mayer, RL Stansifer, and N John Castellan. 1979. Tri-level study of the causes of traffic accidents: final report. Executive summary.Technical Report. Indiana University, Bloomington, Institute for Research in Public Safety.
[50]
Arie P. van den Beukel and Mascha C. van der Voort. 2017. How to assess driver’s interaction with partially automated driving systems – A framework for early concept assessment. Applied Ergonomics 59 (Mar 2017), 302–312. https://doi.org/10.1016/j.apergo.2016.09.005
[51]
Bernhard Wandtner, Nadja Schömig, and Gerald J. Schmidt. 2018. Effects of Non-Driving Related Task Modalities on Takeover Performance in Highly Automated Driving. Human Factors: The Journal of Human Factors and Ergonomics Society 60 (2018), 870 – 881.
[52]
Bradley W. Weaver and Patricia R. Delucia. 2020. A Systematic Review and Meta-Analysis of Takeover Performance During Conditionally Automated Driving.Human factors (2020), 18720820976476.
[53]
Christopher D. Wickens. 2008. Multiple Resources and Mental Workload. Human Factors 50, 3 (2008), 449–455. https://doi.org/10.1518/001872008X288394 arXiv:https://doi.org/10.1518/001872008X288394PMID: 18689052.
[54]
Allan F. Williams and Veronika I. Shabanova. 2003. Responsibility of drivers, by age and gender, for motor-vehicle crash deaths. Journal of Safety Research 34, 5 (2003), 527–531. https://doi.org/10.1016/j.jsr.2003.03.001
[55]
E. Won and Y. Kim. 2016. Stress, the Autonomic Nervous System, and the Immune-kynurenine Pathway in the Etiology of Depression. Current Neuropharmacology 14 (2016), 665 – 673.
[56]
Felix Wulf, Kathrin Zeeb, Maria Rimini-Doring, Marc Arnon, and Frank Gauterin. 2013. Effects of human-machine interaction mechanisms on situation awareness in partly automated driving. IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, 2012–2019. https://doi.org/10.1109/ITSC.2013.6728525
[57]
Sol Hee Yoon and Yong Gu Ji. 2019. Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts. Transportation Research Part F: Traffic Psychology and Behaviour 60 (2019), 620–631. https://doi.org/10.1016/j.trf.2018.11.015
[58]
Mark S. Young, Karel A. Brookhuis, Christopher D. Wickens, and Peter A. Hancock. 2015. State of science: mental workload in ergonomics. Ergonomics 58, 1 (Jan. 2015), 1–17. https://doi.org/10.1080/00140139.2014.956151
[59]
Kathrin Zeeb, A. Buchner, and M. Schrauf. 2016. Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving.Accident; analysis and prevention 92 (2016), 230–9.
Index Terms
- Effect of Obstacle Type and Cognitive Task on Situation Awareness and Takeover Performance in Conditionally Automated Driving
Recommendations
Attention-guiding Takeover Requests for Situation Awareness in Semi-autonomous Driving
HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot InteractionIn semi-autonomous driving (SAE Level-3), the automated driving system allows drivers to focus on their non-driving-related tasks for the majority of the journey. However, when the system faces situations beyond its operational design domain, the drivers ...
Evaluating Effects of Cognitive Load, Takeover Request Lead Time, and Traffic Density on Drivers’ Takeover Performance in Conditionally Automated Driving
AutomotiveUI '20: 12th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsIn conditionally automated driving, drivers engaged in non-driving related tasks (NDRTs) have difficulty taking over control of the vehicle when requested. This study aimed to examine the relationships between takeover performance and drivers’ ...
Takeover quality prediction based on driver physiological state of different cognitive tasks in conditionally automated driving
AbstractIn conditionally automated driving, traffic safety problems would occur if the driver does not properly take over the control authority when the request of automated system arises. Therefore, this study proposes XGBoost learning method ...
Comments
Information & Contributors
Information
Published In
April 2023
288 pages
ISBN:9781450398244
DOI:10.1145/3583961
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
IHM '23
IHM '23: 34th International Francophone Conference on Human-Computer Interaction
April 3 - 6, 2023
TROYES, France
Acceptance Rates
Overall Acceptance Rate 103 of 199 submissions, 52%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 373Total Downloads
- Downloads (Last 12 months)272
- Downloads (Last 6 weeks)42
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in