Functional near-infrared spectroscopy (fNIRS) and Eye tracking for Cognitive Load classification in a Driving Simulator Using Deep Learning
0
🏷️
Sign in to get full access
Overview
- Motion simulators allow researchers to safely study how drivers interact with vehicles.
- Many studies using driving simulator data to predict cognitive load only use two workload levels, leaving a gap in research on employing deep learning to analyze cognitive load, especially in low-light conditions.
- Studies often overlook or focus solely on bright daylight scenarios.
- This study uses functional near-infrared spectroscopy (fNIRS) and eye-tracking data, including fixation duration and gaze direction, during simulated driving tasks in low visibility conditions to understand the correlation between performance and cognitive load.
Plain English Explanation
Driving simulators provide a safe way for researchers to study how people interact with vehicles. However, many studies that use data from driving simulators to try to predict how mentally demanding a driving task is only look at two levels of workload, leaving room for more research using advanced machine learning techniques to analyze cognitive load, especially in challenging low-light conditions. Often, these studies focus only on bright daylight scenarios.
To address this gap, this research project used functional near-infrared spectroscopy (fNIRS) - a brain imaging technique - and data on where people were looking (eye-tracking) during simulated driving tasks in low visibility conditions. The goal was to understand the relationship between driving performance and how mentally demanding the task was.
Technical Explanation
The first step was to statistically analyze the fNIRS and eye-tracking data to identify useful features. The researchers used ANOVA to pinpoint significant channels from the fNIRS signals. They then combined the optimal features from the fNIRS, eye-tracking, and vehicle dynamics data into one set of inputs for a convolutional neural network (CNN) and long short-term memory (LSTM) model to predict variations in cognitive workload.
This proposed CNN-LSTM model achieved 99% accuracy using the neurological data and 89% accuracy using the vehicle dynamics data to predict cognitive load. This indicates the potential for real-time assessment of a driver's mental state and could guide the development of safe, adaptive systems in vehicles.
Critical Analysis
The study provides a novel approach to studying cognitive load during driving by incorporating neurological data from fNIRS and eye-tracking. This multifaceted data collection and analysis method offers a more comprehensive understanding of driver mental workload compared to previous research that has often relied on limited measures.
However, the study is focused on simulated driving in low-visibility conditions, which may not fully capture the complexity and unpredictability of real-world driving. Additionally, the high accuracy rates reported for the predictive model may not generalize well to more diverse driving scenarios or populations. Further research is needed to validate the model's performance in more realistic driving conditions and explore how the insights could be applied to the development of adaptive driver assistance systems.
Conclusion
This study demonstrates the potential of using advanced neuroimaging and eye-tracking data, combined with deep learning techniques, to gain a more nuanced understanding of driver cognitive load, particularly in challenging low-visibility conditions. The high accuracy of the predictive model suggests this approach could enable real-time assessment of mental workload and inform the design of safer, more adaptive vehicle systems that respond to the driver's cognitive state. However, further validation and real-world testing are needed to fully realize the practical applications of this research.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🏷️
0
Functional near-infrared spectroscopy (fNIRS) and Eye tracking for Cognitive Load classification in a Driving Simulator Using Deep Learning
Mehshan Ahmed Khan, Houshyar Asadi, Mohammad Reza Chalak Qazani, Chee Peng Lim, Saied Nahavandi
Motion simulators allow researchers to safely investigate the interaction of drivers with a vehicle. However, many studies that use driving simulator data to predict cognitive load only employ two levels of workload, leaving a gap in research on employing deep learning methodologies to analyze cognitive load, especially in challenging low-light conditions. Often, studies overlook or solely focus on scenarios in bright daylight. To address this gap and understand the correlation between performance and cognitive load, this study employs functional near-infrared spectroscopy (fNIRS) and eye-tracking data, including fixation duration and gaze direction, during simulated driving tasks in low visibility conditions, inducing various mental workloads. The first stage involves the statistical estimation of useful features from fNIRS and eye-tracking data. ANOVA will be applied to the signals to identify significant channels from fNIRS signals. Optimal features from fNIRS, eye-tracking and vehicle dynamics are then combined in one chunk as input to the CNN and LSTM model to predict workload variations. The proposed CNN-LSTM model achieved 99% accuracy with neurological data and 89% with vehicle dynamics to predict cognitive load, indicating potential for real-time assessment of driver mental state and guide designers for the development of safe adaptive systems.
Read more8/14/2024
🏷️
0
Enhancing Cognitive Workload Classification Using Integrated LSTM Layers and CNNs for fNIRS Data Analysis
Mehshan Ahmed Khan, Houshyar Asadi, Mohammad Reza Chalak Qazani, Adetokunbo Arogbonlo, Siamak Pedrammehr, Adnan Anwar, Asim Bhatti, Saeid Nahavandi, Chee Peng Lim
Functional near-infrared spectroscopy (fNIRS) is employed as a non-invasive method to monitor functional brain activation by capturing changes in the concentrations of oxygenated haemoglobin (HbO) and deoxygenated haemo-globin (HbR). Various machine learning classification techniques have been utilized to distinguish cognitive states. However, conventional machine learning methods, although simpler to implement, undergo a complex pre-processing phase before network training and demonstrate reduced accuracy due to inadequate data preprocessing. Additionally, previous research in cog-nitive load assessment using fNIRS has predominantly focused on differ-sizeentiating between two levels of mental workload. These studies mainly aim to classify low and high levels of cognitive load or distinguish between easy and difficult tasks. To address these limitations associated with conven-tional methods, this paper conducts a comprehensive exploration of the im-pact of Long Short-Term Memory (LSTM) layers on the effectiveness of Convolutional Neural Networks (CNNs) within deep learning models. This is to address the issues related to spatial features overfitting and lack of tem-poral dependencies in CNN in the previous studies. By integrating LSTM layers, the model can capture temporal dependencies in the fNIRS data, al-lowing for a more comprehensive understanding of cognitive states. The primary objective is to assess how incorporating LSTM layers enhances the performance of CNNs. The experimental results presented in this paper demonstrate that the integration of LSTM layers with Convolutional layers results in an increase in the accuracy of deep learning models from 97.40% to 97.92%.
Read more7/24/2024
📈
0
Predicting cognitive load in immersive driving scenarios with a hybrid CNN-RNN model
Mehshan Ahmed Khan, Houshyar Asadi, Mohammad Reza Chalak Qazani, Adetokunbo Arogbonlo, Saeid Nahavandi, Chee Peng Lim
One debatable issue in traffic safety research is that cognitive load from sec-ondary tasks reduces primary task performance, such as driving. Although physiological signals have been extensively used in driving-related research to assess cognitive load, only a few studies have specifically focused on high cognitive load scenarios. Most existing studies tend to examine moderate or low levels of cognitive load In this study, we adopted an auditory version of the n-back task of three levels as a cognitively loading secondary task while driving in a driving simulator. During the simultaneous execution of driving and the n-back task, we recorded fNIRS, eye-tracking, and driving behavior data to predict cognitive load at three different levels. To the best of our knowledge, this combination of data sources has never been used before. Un-like most previous studies that utilize binary classification of cognitive load and driving in conditions without traffic, our study involved three levels of cognitive load, with drivers operating in normal traffic conditions under low visibility, specifically during nighttime and rainy weather. We proposed a hybrid neural network combining a 1D Convolutional Neural Network and a Recurrent Neural Network to predict cognitive load. Our experimental re-sults demonstrate that the proposed model, with fewer parameters, increases accuracy from 99.82% to 99.99% using physiological data, and from 87.26% to 92.02% using driving behavior data alone. This significant improvement highlights the effectiveness of our hybrid neural network in accurately pre-dicting cognitive load during driving under challenging conditions.
Read more8/14/2024
🔮
0
Block-As-Domain Adaptation for Workload Prediction from fNIRS Data
Jiyang Wang, Ayse Altay, Senem Velipasalar
Functional near-infrared spectroscopy (fNIRS) is a non-intrusive way to measure cortical hemodynamic activity. Predicting cognitive workload from fNIRS data has taken on a diffuse set of methods. To be applicable in real-world settings, models are needed, which can perform well across different sessions as well as different subjects. However, most existing works assume that training and testing data come from the same subjects and/or cannot generalize well across never-before-seen subjects. Additional challenges imposed by fNIRS data include the high variations in inter-subject fNIRS data and also in intra-subject data collected across different blocks of sessions. To address these issues, we propose an effective method, referred to as the class-aware-block-aware domain adaptation (CABA-DA) which explicitly minimize intra-session variance by viewing different blocks from the same subject same session as different domains. We minimize the intra-class domain discrepancy and maximize the inter-class domain discrepancy accordingly. In addition, we propose an MLPMixer-based model for cognitive load classification. Experimental results demonstrate the proposed model has better performance compared with three different baseline models on three public-available datasets of cognitive workload. Two of them are collected from n-back tasks and one of them is from finger tapping. From our experiments, we also show the proposed contrastive learning method can also improve baseline models we compared with.
Read more5/2/2024