research-article

Language-based multimodal displays for the handover of control in autonomous cars

Authors:

Ioannis Politis,

Stephen Brewster,

Frank PollickAuthors Info & Claims

AutomotiveUI '15: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

Pages 3 - 10

https://doi.org/10.1145/2799250.2799262

Published: 01 September 2015 Publication History

Abstract

This paper presents a first evaluation of multimodal language-based warnings for handovers of control in autonomous cars. A set of possible handover situations varying in urgency is described. A set of multimodal, language-based warnings for these situations is then introduced. All combinations of audio, tactile and visual warnings for handovers were evaluated in terms of perceived urgency, annoyance and alerting effectiveness. Results showed clear recognition of the warning urgency in this new context, as well as low perceived annoyance overall, and higher perceived effectiveness for critical warnings. The time of transition from self-driving to manual mode in the presence of the warnings was then evaluated. Results showed quicker transitions for highly urgent warnings and poor driving performance for unimodal visual warnings. These results provide a novel set of guidelines for an effective transition of control between car and driver in an autonomous vehicle.

References

[1]

Baldwin, C. L. and Moore, C. Perceived Urgency, Alerting Effectiveness and Annoyance of Verbal Collision Avoidance System Messages. HFES Annual Meeting 46, 22 (2002), 1848--1852.

[2]

Baldwin, C. L. Verbal collision avoidance messages during simulated driving: perceived urgency, alerting effectiveness and annoyance. Ergonomics 54, 4 (2011), 328--337.

[3]

Edworthy, J., Hellier, E., Walters, K., Clift-Mathews, W., and Crowther, M. Acoustic, semantic and phonetic influences in spoken warning signal words. Applied Cognitive Psychology 17, 8 (2003), 915--933.

[4]

Geiselman, E. E., Johnson, C. M., and Buck, D. R. Flight Deck Automation: Invaluable Collaborator or Insidious Enabler? Ergonomics in Design: The Quarterly of Human Factors Applications 21, 3 (2013), 22--26.

[5]

Gold, C., Damböck, D., Lorenz, L., and Bengler, K."Take over!" How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, SAGE Publications (2013), 1938--1942.

[6]

Hellier, E., Edworthy, J., Weedon, B., Walters, K., and Adams, A. The Perceived Urgency of Speech Warnings: Semantics versus Acoustics. Human Factors 44, 1 (2002), 1--17.

[7]

Ho, C., Reed, N., and Spence, C. Multisensory In-Car Warning Signals for Collision Avoidance. Human Factors 49, 6 (2007), 1107--1114.

[8]

Koo, J., Kwac, J., Ju, W., Steinert, M., Leifer, L., and Nass, C. Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance. International Journal on Interactive Design and Manufacturing (IJIDeM), (2014), 1--7.

[9]

Kyriakidis, M., Happee, R., and Winter, J. C. F. De. Public opinion on automated driving : Results of an international questionnaire among 5,000 respondents. 2014.

[10]

Lee, J. D., Bricker, D. L., and Hoffman, J. D. A Dynamic Programming Algorithm for Scheduling In-Vehicle Messages. IEEE Transactions on Intelligent Transportation Systems 9, 2 (2008), 226--234.

Digital Library

[11]

Merat, N., Jamson, a. H., Lai, F. F. C., Daly, M., and Carsten, O. M. Transition to manual: Driver behaviour when resuming control from a highly automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour 27, (2014), 274--282.

[12]

National Highway Traffic Safety Administration. Preliminary Statement of Policy Concerning Automated Vehicles. Washington, DC, 2013.

[13]

Naujoks, F., Mai, C., and Neukum, A. The effect of urgency of take-over requests during highly automated driving under distraction conditions. Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics AHFE 2014, (2014), 431--438.

[14]

Politis, I., Brewster, S., and Pollick, F. Evaluating Multimodal Driver Displays under Varying Situational Urgency. CHI 2014, ACM Press (2014), 4067--4076.

Digital Library

[15]

Politis, I., Brewster, S., and Pollick, F. Evaluating Multimodal Driver Displays of Varying Urgency. Automotive UI 2013, ACM Press (2013), 92--99.

Digital Library

[16]

Politis, I., Brewster, S., and Pollick, F. Speech Tactons Improve Speech Warnings for Drivers. Automotive UI 2014, ACM Press (2014), 1--8.

Digital Library

[17]

Politis, I., Brewster, S., and Pollick, F. To Beep or Not to Beep? Comparing Abstract versus Language-Based Multimodal Driver Displays. CHI 2015, (2015), In Press.

Digital Library

[18]

SAE J3016. Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems. 2014.

[19]

SAE.Its In-Vehicle Message Priority Standard. J2395, (2002), 1--7.

[20]

Warnock, D., Mcgee-lennon, M., and Brewster, S. The Role of Modality in Notification Performance. (2011), 572--588.

Digital Library

[21]

De Winter, J. C. F., Happee, R., Martens, M. H., and Stanton, N. A. Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transportation Research Part F: Traffic Psychology and Behaviour 27, (2014), 196--217.

Cited By

Jin LLiu XGuo BHan ZWang YCao YYang XShi J(2025)Impact of non-driving related task types, request modalities, and automation on driver takeover: A meta-analysisSafety Science10.1016/j.ssci.2024.106704181(106704)Online publication date: Jan-2025
https://doi.org/10.1016/j.ssci.2024.106704
Huang ADerakhshan SMadrid-Carvajal JNosrat Nezami FWächter MPipa GKönig P(2024)Enhancing Safety in Autonomous Vehicles: The Impact of Auditory and Visual Warning Signals on Driver Behavior and Situational AwarenessVehicles10.3390/vehicles60300766:3(1613-1636)Online publication date: 8-Sep-2024
https://doi.org/10.3390/vehicles6030076
Guo HQiu HZhou YDeng Y(2024)Analysis of Influencing Factors of Level 3 Automated Vehicle Takeover: A Literature ReviewSustainability10.3390/su1619834516:19(8345)Online publication date: 25-Sep-2024
https://doi.org/10.3390/su16198345
Show More Cited By

Index Terms

Language-based multimodal displays for the handover of control in autonomous cars
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Sound-based input / output
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
      2. Sound-based input / output
    2. Interaction techniques
      1. Auditory feedback

Recommendations

Priming Drivers before Handover in Semi-Autonomous Cars
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Semi-autonomous vehicles occasionally require control to be handed over to the driver in situations where the vehicle is unable to operate safely. Currently, such handover requests require the driver to take control almost instantaneously. We ...
Using Multimodal Displays to Signify Critical Handovers of Control to Distracted Autonomous Car Drivers

Until full autonomy is achieved in cars, drivers will still be expected to take over control of driving, and critical warnings will be essential. This paper presents a comparison of abstract versus language-based multimodal warnings signifying handovers ...
To Beep or Not to Beep?: Comparing Abstract versus Language-Based Multimodal Driver Displays
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Multimodal displays are increasingly being utilized as driver warnings. Abstract warnings, without any semantic association to the signified event, and language-based warnings are examples of such displays. This paper presents a first comparison between ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AutomotiveUI '15: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 2015

338 pages

ISBN:9781450337366

DOI:10.1145/2799250

General Chair:
Gary Burnett
University of Nottingham, UK
,
Program Chairs:
Joe Gabbard
Virginia Tech
,
Paul Green
University of Michigan Transportation Research Institute (UMTRI)
,
Sebastian Osswald
Technische Universitat Munchen (TUM), Germany
,
Publications Chairs:
Ayse Eren,
Vicki Antrobus

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Freescale Semiconductor Inc., Automotive Microcontroller Product Group

Conference

AutomotiveUI '15

AutomotiveUI '15: The 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications

September 1 - 3, 2015

Nottingham, United Kingdom

Acceptance Rates

AutomotiveUI '15 Paper Acceptance Rate 38 of 80 submissions, 48%;

Overall Acceptance Rate 248 of 566 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

143
Total Citations
View Citations
1,557
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)9

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jin LLiu XGuo BHan ZWang YCao YYang XShi J(2025)Impact of non-driving related task types, request modalities, and automation on driver takeover: A meta-analysisSafety Science10.1016/j.ssci.2024.106704181(106704)Online publication date: Jan-2025
https://doi.org/10.1016/j.ssci.2024.106704
Huang ADerakhshan SMadrid-Carvajal JNosrat Nezami FWächter MPipa GKönig P(2024)Enhancing Safety in Autonomous Vehicles: The Impact of Auditory and Visual Warning Signals on Driver Behavior and Situational AwarenessVehicles10.3390/vehicles60300766:3(1613-1636)Online publication date: 8-Sep-2024
https://doi.org/10.3390/vehicles6030076
Guo HQiu HZhou YDeng Y(2024)Analysis of Influencing Factors of Level 3 Automated Vehicle Takeover: A Literature ReviewSustainability10.3390/su1619834516:19(8345)Online publication date: 25-Sep-2024
https://doi.org/10.3390/su16198345
Elshafei ARomano DFahim I(2024)The Effect of Directional Tactile Memory of the Back of the User on Reaction Time and AccuracyElectronics10.3390/electronics1313248213:13(2482)Online publication date: 25-Jun-2024
https://doi.org/10.3390/electronics13132482
Han DLiu JYang XRomo AHorrey WTilbury DZhou FRobert LMolnar L(2024)Toward Integrated Takeover Performance Measurement: Validation of Fréchet Distance as a Takeover Performance MetricProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/10711813241266831Online publication date: 29-Aug-2024
https://doi.org/10.1177/10711813241266831
Srour Zreik RGoodge THarvey MBrewster S(2024)Where's my TOR?: Evaluating the Effect of Take-Over Request Source on Older Drivers' Control Transition in Level 3 CarsProceedings of the ACM on Human-Computer Interaction10.1145/36765338:MHCI(1-25)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676533
Sun QYao JCheng PHou S(2024)How can level of trust in conditional driving automation vehicles be improved? Road condition feedback strategies and the mediating role of anxietyBehaviour & Information Technology10.1080/0144929X.2024.2420183(1-17)Online publication date: 28-Oct-2024
https://doi.org/10.1080/0144929X.2024.2420183
Kim Svan Egmond RHappee R(2024)How manoeuvre information via auditory (spatial and beep) and visual UI can enhance trust and acceptance in automated drivingTransportation Research Part F: Traffic Psychology and Behaviour10.1016/j.trf.2023.11.007100(22-36)Online publication date: Jan-2024
https://doi.org/10.1016/j.trf.2023.11.007
Becker SBrandenburg SThüring M(2024)Driver-initiated take-overs during critical evasion maneuvers in automated drivingAccident Analysis & Prevention10.1016/j.aap.2023.107362194(107362)Online publication date: Jan-2024
https://doi.org/10.1016/j.aap.2023.107362
Yang WWu ZTang JLiang Y(2023)Assessing the Effects of Modalities of Takeover Request, Lead Time of Takeover Request, and Traffic Conditions on Takeover Performance in Conditionally Automated DrivingSustainability10.3390/su1509727015:9(7270)Online publication date: 27-Apr-2023
https://doi.org/10.3390/su15097270
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents