skip to main content
research-article

Takeover quality prediction based on driver physiological state of different cognitive tasks in conditionally automated driving

Published: 01 August 2023 Publication History

Abstract

In conditionally automated driving, traffic safety problems would occur if the driver does not properly take over the control authority when the request of automated system arises. Therefore, this study proposes XGBoost learning method considering risk potential field to predict the takeover quality in conditionally automated driving under different levels of cognitive non-driving related tasks (NDRTs). Thirty participants drive on two experimental conditions: manual driving is following an automated driving during which the driver is asked to perform NDRTs. Drivers’ physiological features of different cognitive states are exploited to model multi-level takeover quality prediction. This investigation also gives an insight into the main effects of the selected prediction variables on the takeover quality. The proposed model performance within different time windows is assessed using multiple evaluation metrics and compared with other methods. Results show that the prediction accuracy within the time windows of 7–10 s, 5–7 s, 3–5 s and 1–3 s is 0.87, 0.85, 0.85 and 0.90, respectively. The XGBoost model has the best performance of different time windows under each level of takeover quality compared to the other three machine learning models. Our findings can effectively predict the takeover quality and assist automated driving safely in interactions between drivers and automated vehicles.

References

[1]
F. Naujoks, C. Purucker, A. Neukum, Secondary task engagement and vehicle automation – comparing the effects of different automation levels in an on-road experiment, Transp. Res. Part F Traffic Psychol. Behav. 38 (2016) 67–82,.
[2]
M. Körber, C. Gold, D. Lechner, K. Bengler, The influence of age on the take-over of vehicle control in highly automated driving, Transp. Res. Part F Traffic Psychol. Behav. 39 (2016) 19–32,.
[3]
D. Miller, M. Johns, H. Ive, N. Gowda, D. Sirkin, S. Sibi, B. Mok, S. Aich, W. Ju, Exploring Transitional Automation with New and Old Drivers, 2016. https://doi.org/10.4271/2016-01-1442.
[4]
H. Clark, J. Feng, Age differences in the takeover of vehicle control and engagement in non-driving-related activities in simulated driving with conditional automation, Accid. Anal. Prev. 106 (2017) 468–479,.
[5]
P. Bazilinskyy, J.C.F. de Winter, Analyzing crowdsourced ratings of speech-based take-over requests for automated driving, Appl. Ergon. 64 (2017) 56–64,.
[6]
S.M. Petermeijer, S. Cieler, J.C.F. de Winter, Comparing spatially static and dynamic vibrotactile take-over requests in the driver seat, Accid. Anal. Prev. 99 (2017) 218–227,.
[7]
S.H. Yoon, Y.W. Kim, Y.G. Ji, The effects of takeover request modalities on highly automated car control transitions, Accid. Anal. Prev. 123 (2019) 150–158,.
[8]
E. Dogan, M.-C. Rahal, R. Deborne, P. Delhomme, A. Kemeny, J. Perrin, Transition of control in a partially automated vehicle: effects of anticipation and non-driving-related task involvement, Transp. Res. Part F Traffic Psychol. Behav. 46 (2017) 205–215,.
[9]
F. Naujoks, S. Höfling, C. Purucker, K. Zeeb, From partial and high automation to manual driving: Relationship between non-driving related tasks, drowsiness and take-over performance, Accid. Anal. Prev. 121 (2018) 28–42,.
[10]
A. Heenan, C.M. Herdman, M.S. Brown, N. Robert, Effects of conversation on situation awareness and working memory in simulated driving, Hum. Factors J. Hum. Factors Ergon. Soc. 56 (2014) 1077–1092,.
[11]
J. Radlmayr, C. Gold, L. Lorenz, M. Farid, K. Bengler, How traffic situations and non-driving related tasks affect the take-over quality in highly automated driving, Proc. Hum. Factors Ergon. Soc. Annu. Meet. 58 (2014) 2063–2067,.
[12]
W. Kim, E. Jeon, G. Kim, D. Yeo, S. Kim, Take-over requests after waking in autonomous vehicles, Appl. Sci. 12 (2022) 1438,.
[13]
H. Jeong, Z. Kang, Y. Liu, Driver glance behaviors and scanning patterns: applying static and dynamic glance measures to the analysis of curve driving with secondary tasks, Hum. Factors Ergon. Manuf. 29 (2019) 437–446,.
[14]
A. Feldhutter, A. Ruhl, A. Feierle, K. Bengler, The effect of fatigue on take-over performance in urgent situations in conditionally automated driving, in: 2019 IEEE Intell. Transp. Syst. Conf. ITSC 2019. (2019) 1889–1894. https://doi.org/10.1109/ITSC.2019.8917183.
[15]
O. Jarosch, H. Bellem, K. Bengler, Effects of task-induced fatigue in prolonged conditional automated driving, Https://Doi.Org/10.1177/0018720818816226. 61 (2019) 1186–1199,.
[16]
K. Zeeb, A. Buchner, M. Schrauf, What determines the take-over time? An integrated model approach of driver take-over after automated driving, Accid. Anal. Prev. 78 (2015) 212–221,.
[17]
A. Alsaid, J.D. Lee, M. Price, Moving into the loop: an investigation of drivers’ steering behavior in highly automated vehicles, Hum. Factors. 62 (2020) 671–683,.
[18]
M. Bueno, E. Dogan, F.H. Selem, E. Monacelli, S. Boverie, A. Guillaume, How different mental workload levels affect the take-over control after automated driving, IEEE Conf. Intell. Transp. Syst. Proc., ITSC. (2016) 2040–2045,.
[19]
Q. Li, L. Hou, Z. Wang, W. Wang, C. Zeng, Q. Yuan, B. Cheng, Drivers’ visual-distracted take-over performance model and its application on adaptive adjustment of time budget, Accid. Anal. Prev. 154 (2021),.
[20]
Q. Lin, S. Li, X. Ma, G. Lu, Understanding take-over performance of high crash risk drivers during conditionally automated driving, Accid. Anal. Prev. 143 (2020),.
[21]
Z. Lu, X. Coster, J. de Winter, How much time do drivers need to obtain situation awareness? A laboratory-based study of automated driving, Appl. Ergon. 60 (2017) 293–304,.
[22]
C. Gold, D. Damböck, L. Lorenz, K. Bengler, Take over! How long does it take to get the driver back into the loop?, Proc. Hum. Factors Ergon. Soc. (2013) 1938–1942,.
[23]
C. Gold, R. Happee, K. Bengler, Modeling take-over performance in level 3 conditionally automated vehicles, Accid. Anal. Prev. 116 (2018) 3–13,.
[24]
R. Happee, C. Gold, J. Radlmayr, S. Hergeth, K. Bengler, Take-over performance in evasive manoeuvres, Accid. Anal. Prev. 106 (2017) 211–222,.
[25]
K. Wiedemann, F. Naujoks, J. Wörle, R. Kenntner-Mabiala, Y. Kaussner, A. Neukum, Effect of different alcohol levels on take-over performance in conditionally automated driving, Accid. Anal. Prev. 115 (2018) 89–97,.
[26]
A. Lotz, N. Russwinkel, E. Wohlfarth, Response times and gaze behavior of truck drivers in time critical conditional automated driving take-overs, Transp. Res. Part F Traffic Psychol. Behav. 64 (2019) 532–551,.
[27]
S.H. Yoon, S.C. Lee, Y.G. Ji, Modeling takeover time based on non-driving-related task attributes in highly automated driving, Appl. Ergon. 92 (2021),.
[28]
C. Braunagel, W. Rosenstiel, E. Kasneci, Ready for take-over? A new driver assistance system for an automated classification of driver take-over readiness, IEEE Intell. Transp. Syst. Mag. 9 (2017) 10–22,.
[29]
N. Deo, M.M. Trivedi, Looking at the driver/rider in autonomous vehicles to predict take-over readiness, IEEE Trans. Intell. Veh. 5 (2020) 41–52,.
[30]
E. Pakdamanian, S. Sheng, S. Baee Deeptake, Prediction of driver takeover behavior using multimodal data, Conf. Hum. Factors Comput. Syst. - Proc. (2021) 1–14,.
[31]
N. Du, F. Zhou, E.M. Pulver, D.M. Tilbury, L.P. Robert, A.K. Pradhan, X.J. Yang, Predicting driver takeover performance in conditionally automated driving, Accid. Anal. Prev. 148 (2020) 1–11,.
[32]
D. Girardi, F. Lanubile, N. Novielli, Emotion detection using noninvasive low cost sensors, 2017 7th Int. Conf. Affect. Comput. Intell. Interact. ACII 2017. 2018-Janua (2018) 125–130. https://doi.org/10.1109/ACII.2017.8273589.
[33]
C.Y. Liao, R.C. Chen, S.K. Tai, Hendry, Using single point brain wave instrument to explore and verification of music frequency, in: Proc. - 2017 Int. Conf. Innov. Creat. Inf. Technol. Comput. Intell. IoT, ICITech 2017. 2018-Janua (2018) 1–6. https://doi.org/10.1109/INNOCIT.2017.8319142.
[34]
C.L. Haohan Yang, W.U. Jingda, Hu Zhongxu, Real-time driver cognitive workload recognition: attention-enabled learning with multimodal information fusion, IEEE Trans. Ind. Electron. (2023),.
[35]
M.A. Almogbel, A.H. Dang, W. Kameyama, Cognitive workload detection from raw EEG-signals of vehicle driver using deep learning, Int. Conf. Adv. Commun. Technol. ICACT. (2019) 1167–1172. https://doi.org/10.23919/ICACT.2019.8702048.
[36]
F. Weidner, W. Broll, Stereoscopic 3D dashboards: An investigation of performance, workload, and gaze behavior during take-overs in semi-autonomous driving, Pers. Ubiquitous Comput. 26 (2020) 697–719,.
[37]
S.H. Yoon, Y.G. Ji, Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts, Transp. Res. Part F Traffic Psychol. Behav. 60 (2019) 620–631,.
[38]
P. Hang, C. Lv, C. Huang, J. Cai, Z. Hu, Y. Xing, An integrated framework of decision making and motion planning for autonomous vehicles considering social behaviors, IEEE Trans. Veh. Technol. 69 (2020) 14458–14469,.
[39]
T. Chen, C. Guestrin, XGBoost: A Scalable Tree Boosting System, in: Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., Association for Computing Machinery, New York, NY, USA, 2016, pp. 785–794. https://doi.org/10.1145/2939672.2939785.
[40]
A. Eriksson, N.A. Stanton, Takeover time in highly automated vehicles: noncritical transitions to and from manual control, Hum. Factors. 59 (2017) 689–705,.
[41]
J. Ayoub, X.J. Yang, F. Zhou, Modeling dispositional and initial learned trust in automated vehicles with predictability and explainability, Transp. Res. Part F Traffic Psychol. Behav. 77 (2021) 102–116,.
[42]
J. Zhu, Y. Ma, Y. Lou, Multi-vehicle interaction safety of connected automated vehicles in merging area: a real-time risk assessment approach, Accid. Anal. Prev. 166 (2022),.
[43]
C. Fu, T. Sayed, Bayesian dynamic extreme value modeling for conflict-based real-time safety analysis, Anal. Methods Accid. Res. 34 (2022),.
[44]
J. Wang, J. Wu, X. Zheng, D. Ni, K. Li, Driving safety field theory modeling and its application in pre-collision warning system, Transp. Res. Part C Emerg. Technol. 72 (2016) 306–324,.
[45]
Y. Wang, B. Reimer, J. Dobres, B. Mehler, The sensitivity of different methodologies for characterizing drivers’ gaze concentration under increased cognitive demand, Transp. Res. Part F Traffic Psychol. Behav. 26 (2014) 227–237,.

Cited By

View all

Index Terms

  1. Takeover quality prediction based on driver physiological state of different cognitive tasks in conditionally automated driving
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image Advanced Engineering Informatics
          Advanced Engineering Informatics  Volume 57, Issue C
          Aug 2023
          1572 pages

          Publisher

          Elsevier Science Publishers B. V.

          Netherlands

          Publication History

          Published: 01 August 2023

          Author Tags

          1. Conditionally automated driving
          2. Takeover quality prediction
          3. Cognitive tasks
          4. Multimodal physiological features
          5. XGBoost and risk potential field

          Qualifiers

          • Research-article

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)0
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 26 Jan 2025

          Other Metrics

          Citations

          Cited By

          View all

          View Options

          View options

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media