skip to main content
research-article

EEG spatial inter-channel connectivity analysis: : A GCN-based dual stream approach to distinguish mental fatigue status

Published: 01 November 2024 Publication History

Abstract

Mental fatigue is defined as a decline in the ability and efficiency of mental activities. A lot of research suggests that the transition from alertness to fatigue is accompanied by alterations in correlation patterns among various brain regions. However, conventional methods for detecting mental fatigue seldom emphases inter-channel connectivity in the spatial domain. To fill this gap, this paper explores the spatial inter-channel connectivity in alertness and fatigue, employing spectral graph convolutional networks (GCN) for mental fatigue detection. We utilized Pearson correlation coefficients (PCC) to establish temporal connections and magnitude-squared coherence (MSC) for spectral connections. Topological features of the brain network were then analysed. To enhance the learning of spatial inter-channel connectivity, a dual-graph strategy transforms edge features into node features, serving as inputs to the spectral GCN. By simultaneously learning PCC and MSC features, the model results indicate significant differences in some brain network characteristics between alert and fatigue states. It confirms that the synchronicity of brain operations differs in the alert state compared to mental fatigue, and indicates that fatigue states can influence correlation patterns among different brain regions. Our approach is evaluated on a self-designed experimental dataset containing 7 subjects, demonstrating a classification accuracy of 89.59 % in group-level experiments and 95.24 % at the subject level. Additionally, on the public dataset SEED-VIG containing 23 subjects, our method achieves an accuracy of 86.58 %. In summary, this paper proposes a neural network approach based on a dynamic functional connectivity network. The network integrates both temporal and spectral connections with the goal of simultaneously learning spatial inter-channel connectivity in time and frequency domains. This effectively accomplishes fatigue state detection, highlighting that fatigue significantly influences correlations among different brain regions.

Highlights

Mental fatigue status features vary with channels, thus requiring model adaptation.
Spectral and temporal connections reflect the EEG properties within individuals.
Graph convolutional network with dual transformation learns feature representation.

References

[1]
R. Xu, et al., How physical activities affect mental fatigue based on EEG energy, connectivity, and complexity, Front Neurol 9 (2018) 915.
[2]
M.A. Boksem, T.F. Meijman, M.M. Lorist, Mental fatigue, motivation and action monitoring, Biol Psychol 72 (2) (2006) 123–132.
[3]
S. Soares, T. Monteiro, A. Lobo, A. Couto, L. Cunha, S. Ferreira, Analyzing driver drowsiness: from causes to effects, Sustainability 12 (5) (2020) 1971.
[4]
A. Shahid, K. Wilkinson, S. Marcu, C.M. Shapiro, “fatigue severity scale (FSS),” in STOP , Springer, THAT and One Hundred Other Sleep Scales, 2011, pp. 167–168.
[5]
N. Crichton, Visual analogue scale (VAS), J Clin Nurs 10 (5) (2001) 706.
[6]
A. Shahid, K. Wilkinson, S. Marcu, C.M. Shapiro, “Fatigue assessment scale (FAS),” in STOP , Springer, THAT and One Hundred Other Sleep Scales, 2011, pp. 161–162.
[7]
G. Sikander, S. Anwar, Driver fatigue detection systems: a review, IEEE Trans Intell Transp Syst 20 (6) (2018) 2339–2352.
[8]
Q. Zhuang, Z. Kehua, J. Wang, Q. Chen, Driver fatigue detection method based on eye states with pupil and iris segmentation, IEEE Access 8 (2020) 173440–173449.
[9]
M. Saradadevi, P. Bajaj, Driver fatigue detection using mouth and yawning analysis, Int J Comput Sci Net 8 (6) (2008) 183–188.
[10]
F. Zhang, J. Su, L. Geng, and Z. Xiao, "Driver fatigue detection based on eye state recognition," in 2017 International Conference on Machine Vision and Information Technology (CMVIT), 2017: IEEE, pp. 105–110.
[11]
L.J. Trejo, K. Kubitz, R. Rosipal, R.L. Kochavi, L.D. Montgomery, EEG-based estimation and classification of mental fatigue, Psychology 6 (05) (2015) 572.
[12]
S. Huang, J. Li, P. Zhang, W. Zhang, Detection of mental fatigue state with wearable ECG devices, Int J Med Inform 119 (2018) 39–46.
[13]
M. Kołodziej, et al., Fatigue detection caused by office work with the use of EOG signal, IEEE Sens J 20 (24) (2020) 15213–15223.
[14]
A. Murata, A. Uetake, Y. Takasawa, Evaluation of mental fatigue using feature parameter extracted from event-related potential, Int J Ind Ergon 35 (8) (2005) 761–770.
[15]
R. Bose, H. Wang, A. Dragomir, N.V. Thakor, A. Bezerianos, J. Li, Regression-based continuous driving fatigue estimation: toward practical implementation, IEEE T Cogn Dev Syst 12 (2) (2019) 323–331.
[16]
Y. Xiong, J. Gao, Y. Yang, X. Yu, W. Huang, Classifying driving fatigue based on combined entropy measure using EEG signals, Int J Control Autom 9 (3) (2016) 329–338.
[17]
Z. Gao, et al., EEG-based spatio–temporal convolutional neural network for driver fatigue evaluation, IEEE T Neur Net Lear 30 (9) (2019) 2755–2763.
[18]
M. Hajinoroozi, Z. Mao, T.-P. Jung, C.-T. Lin, Y. Huang, EEG-based prediction of driver’s cognitive performance by deep convolutional neural network, Signal Process Image Commun 47 (2016) 549–555.
[19]
K. Yue, D. Wang, EEG-based 3D visual fatigue evaluation using CNN, Electronics 8 (11) (2019) 1208.
[20]
H. Jia, Z. Xiao, P. Ji, End-to-end fatigue driving EEG signal detection model based on improved temporal-graph convolution network, Comput Biol Med 152 (2023).
[21]
G. Bao, et al., Linking multi-layer dynamical GCN with style-based recalibration CNN for EEG-based emotion recognition, Front Neurorobot 16 (2022).
[22]
X. Che, Y. Zheng, X. Chen, S. Song, S. Li, Decoding color visual working memory from EEG signals using graph convolutional neural networks, Int J Neural Syst 32 (02) (2022) 2250003.
[23]
Y. Hou, et al., Deep feature mining via the attention-based bidirectional long short term memory graph convolutional neural network for human motor imagery recognition, Front Bioeng Biotechnol 9 (2022).
[24]
Z. Lin, T. Qiu, P. Liu, L. Zhang, S. Zhang, Z. Mu, Fatigue driving recognition based on deep learning and graph neural network, Biomed Signal Process Control 68 (2021).
[25]
W. Zhang, F. Wang, S. Wu, Z. Xu, J. Ping, Y. Jiang, Partial directed coherence based graph convolutional neural networks for driving fatigue detection, Rev Sci Instrum 91 (7) (2020).
[26]
H. Wang, et al., Dynamic reorganization of functional connectivity unmasks fatigue related performance declines in simulated driving, IEEE Trans Neural Syst Rehabil Eng 28 (8) (2020) 1790–1799.
[27]
S. Zou, T. Qiu, P. Huang, H. Luo, X. Bai, The functional brain network based on the combination of shortest path tree and its application in fatigue driving state recognition and analysis of the neural mechanism of fatigue driving, Biomed Signal Process Control 62 (2020).
[28]
I. Cohen, et al., Pearson correlation coefficient, Noise reduction in speech processing (2009) 1–4.
[29]
G. Carter, C. Knapp, A. Nuttall, Estimation of the magnitude-squared coherence function via overlapped fast Fourier transform processing, IEEE Trans Audio Electroacoust 21 (4) (1973) 337–344.
[30]
F. Monti, O. Shchur, A. Bojchevski, O. Litany, S. Günnemann, and M. M. Bronstein, "Dual-primal graph convolutional networks," arXiv preprint arXiv:1806.00770 , 2018.
[31]
Y. Wang, Y. Huang, B. Gu, S. Cao, D. Fang, Identifying mental fatigue of construction workers using EEG and deep learning, Autom Constr 151 (2023).
[32]
I. Mehmood, et al., Deep learning-based construction equipment operators’ mental fatigue classification using wearable EEG sensor data, Adv Eng Inform 56 (2023).
[33]
C. Ye, Z. Yin, M. Zhao, Y. Tian, Z. Sun, Identification of mental fatigue levels in a language understanding task based on multi-domain EEG features and an ensemble convolutional neural network, Biomed Signal Process Control 72 (2022).
[34]
M. Su, W. Li, F. Peng, W. Zhou, R. Zhang, and Y. Wen, "EEG-Based Mental Fatigue Detection Using CNN-LSTM," in 2022 16th ICME International Conference on Complex Medical Engineering (CME), 2022: IEEE, pp. 302–305.
[35]
H. Wang, L. Xu, A. Bezerianos, C. Chen, Z. Zhang, Linking attention-based multiscale CNN with dynamical GCN for driving fatigue detection, IEEE Trans Instrum Meas 70 (2020) 1–11.
[36]
R. Li, Z. Lan, J. Cui, O. Sourina, and L. Wang, "EEG-based recognition of driver state related to situation awareness using graph convolutional networks," in 2020 International Conference on Cyberworlds (CW), 2020: IEEE, pp. 180–187.
[37]
Z. Zhang, Z. Zhao, H. Qu, C.a. Liu, L. Pang, A mental workload classification method based on gcn modified by squeeze-and-excitation residual, Mathematics 11 (5) (2023) 1189.
[38]
J.D. Cohen, et al., Temporal dynamics of brain activation during a working memory task, Nature 386 (6625) (1997) 604–608.
[39]
C. Jackson, The Chalder fatigue scale (CFQ 11), Occup Med 65 (1) (2015) 86.
[40]
R. Suppiah, A. Sharma, N. Kim, and K. Abidi, "A Novel Event-Related Desynchronization/Synchronization with Gamma Peak EEG model for Motor State Identification," in 2021 International Conference on Computational Science and Computational Intelligence (CSCI), 2021: IEEE, pp. 1169–1175.
[41]
W.L. Zheng, B.L. Lu, A Multimodal Approach to Estimating Vigilance Using EEG and Forehead EOG, J Neural Eng 14 (2) (2016).
[42]
J. Benesty, J. Chen, Y. Huang, I. Cohen, Pearson correlation coefficient, in: Noise reduction in speech processing, Springer, 2009, pp. 1–4.
[43]
M. Rubinov, O. Sporns, Complex network measures of brain connectivity: uses and interpretations, Neuroimage 52 (3) (2010) 1059–1069.
[44]
R.F. Woolson, Wilcoxon signed-rank test, Encyclopedia of Biostatistics 8 (2005).
[45]
M. Defferrard, X. Bresson, P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral filtering, Adv Neural Inf Proces Syst (2016) 29.
[46]
D.I. Shuman, S.K. Narang, P. Frossard, A. Ortega, P. Vandergheynst, The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains, IEEE Signal Process Mag 30 (3) (2013) 83–98.
[47]
D.K. Hammond, P. Vandergheynst, R.J.A. Gribonval, C.H. Analysis, Wavelets on graphs via spectral graph theory Applied and Computational Harmonic, Analysis 30 (2) (2011) 129–150.
[48]
I.S. Dhillon, Y. Guan, B. Kulis, Weighted graph cuts without eigenvectors a multilevel approach, IEEE Trans Pattern Anal Mach Intell 29 (11) (2007) 1944–1957.
[49]
Z. Wang, Y. Zhao, Y. He, J. Zhang, Phase lag index-based graph attention networks for detecting driving fatigue, Rev Sci Instrum 92 (9) (2021).
[50]
R. Zheng, Z. Wang, Y. He, J. Zhang, EEG-based brain functional connectivity representation using amplitude locking value for fatigue-driving recognition, Cogn Neurodyn 16 (2) (2022) 325–336.
[51]
G. Tononi, An information integration theory of consciousness, BMC Neurosci 5 (1) (2004) 1–22.
[52]
A.A. Fingelkurts, A.A. Fingelkurts, S. Kähkönen, Functional connectivity in the brain—is it an elusive concept?, Neurosci Biobehav Rev 28 (8) (2005) 827–836.
[53]
S. Kar, A. Routray, Effect of sleep deprivation on functional connectivity of EEG channels, IEEE Trans Syst Man Cybern Syst Hum 43 (3) (2012) 666–672.
[54]
H. Wang, et al., Driving fatigue recognition with functional connectivity based on phase synchronization, IEEE Trans Cogn Dev Syst 13 (3) (2020) 668–678.
[55]
M.M. Lorist, E. Bezdan, et al., The influence of mental fatigue and motivation on neural network dynamics; an EEG coherence study, Brain Res 1270 (2009) 95–106.
[56]
G.N. Dimitrakopoulos, et al., Functional connectivity analysis of mental fatigue reveals different network topological alterations between driving and vigilance tasks, IEEE Trans Neural Syst Rehabil Eng 26 (4) (2018) 740–749.
[57]
J. Chen, S. Wang, E. He, et al., Two-dimensional phase lag index image representation of electroencephalography for automated recognition of driver fatigue using convolutional neural network, Expert Syst Appl 191 (2022) 116339.
[58]
L. Zhang, H. Song, N. Aletras, et al., Graph node-feature convolution for representation learning, 2018, arXiv preprint arXiv:1812.00086.
[59]
R. Abel, I. Benami, Y. Louzoun, Topological based classification using graph convolutional networks, 2019, arXiv preprint arXiv:1911.06892.
[60]
Y. Hou, S. Jia, X. Lun, et al., GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals, 2020, arXiv preprint arXiv:2006.08924.
[61]
W. Zhang, F. Wang, S. Wu, Z. Xu, et al., Partial directed coherence based graph convolutional neural networks for driving fatigue detection, Rev Sci Instrum 91 (7) (2020).
[62]
T. Song, W. Zheng, P. Song, et al., EEG emotion recognition using dynamical graph convolutional neural networks, IEEE Trans Affect Comput 11 (3) (2018) 532–541.
[63]
S. Yan, Y. Xiong, D. Lin, “spatial temporal graph convolutional networks for skeleton-based action recognition,” in thirty-second AAAI conference on artificial intelligence , 2018.
[64]
Z. Jia, Y. Lin, J. Wang, et al., GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification," in, IJCAI (2020) 1324–1330.
[65]
Z. Jia, Y. Lin, J. Wang, et al., Multi-view spatial-temporal graph convolutional networks with domain generalization for sleep stage classification, IEEE Trans Neural Syst Rehabil Eng 29 (2021) 1977–1986.
[66]
M.Z. Baig, N. Aslam, H.P. Shum, Filtering techniques for channel selection in motor imagery EEG applications: a survey, Artif Intell Rev 53 (2) (2020) 1207–1232.
[67]
J. Li, T. Zhang, H. Tian, S. Jin, M. Fardad, R. Zafarani, Graph sparsification with graph convolutional networks, Int J Data Sci Anal (2022) 1–14.
[68]
D. Wen et al., "RFC-HyPGCN: A runtime sparse feature compress accelerator for skeleton-based GCNs action recognition model with hybrid pruning," in 2021 IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors (ASAP), 2021: IEEE, pp. 33–40.
[69]
T.A. Taz, S. Arslanturk, Unveiling genetic variant-level biomarkers for aggressive prostate cancer, Inform Med Unlocked 43 (2023).
[70]
S.A. Munj, T.A. Taz, S. Arslanturk, E.I. Heath, Biomarker-driven drug repurposing on biologically similar cancers with DNA-repair deficiencies, Front Genet 13 (2022) 1015531.
[71]
Y. Tran, A. Craig, R. Craig, R. Chai, H. Nguyen, The influence of mental fatigue on brain activity: evidence from a systematic review with meta-analyses, Psychophysiology 57 (5) (2020).

Index Terms

  1. EEG spatial inter-channel connectivity analysis: A GCN-based dual stream approach to distinguish mental fatigue status
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image Artificial Intelligence in Medicine
          Artificial Intelligence in Medicine  Volume 157, Issue C
          Nov 2024
          404 pages

          Publisher

          Elsevier Science Publishers Ltd.

          United Kingdom

          Publication History

          Published: 01 November 2024

          Author Tags

          1. Mental fatigue detection
          2. Graph convolutional network
          3. Functional connectivity networks
          4. Pearson correlation coefficient
          5. Magnitude-squared coherence
          6. Dual-graph

          Qualifiers

          • Research-article

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 0
            Total Citations
          • 0
            Total Downloads
          • Downloads (Last 12 months)0
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 06 Feb 2025

          Other Metrics

          Citations

          View Options

          View options

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media