CLARE: Cognitive Load Assessment in REaltime with Multimodal Data
0
📊
Sign in to get full access
Overview
- The researchers present a new multimodal dataset called CLARE (Cognitive Load Assessment in REaltime) for studying cognitive load.
- The dataset contains physiological and eye-tracking data from 24 participants, along with self-reported cognitive load scores as ground truth.
- The dataset includes four modalities: Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking.
- Participants performed a computer-based task (MATB-II) with varying levels of complexity, reporting their cognitive load every 10 seconds.
- The researchers provide benchmark results using machine learning and deep learning models for binary classification of cognitive load.
Plain English Explanation
The researchers have created a new dataset that could help us better understand how people's minds work under different levels of mental effort or "cognitive load." The dataset includes information from 24 people who did a computer task that got more and less difficult over time. As they were doing the task, the researchers measured things like the participants' heart activity, sweat levels, brain waves, and eye movements. The participants also reported how mentally demanding the task felt to them every 10 seconds.
The researchers hope this dataset will be useful for developing systems that can automatically detect when someone is experiencing high cognitive load, perhaps by analyzing their physiological signals and gaze patterns. This could have applications in areas like assessing operator performance or monitoring mental workload in real-time.
Technical Explanation
The CLARE dataset contains multimodal data from 24 participants who completed a computer-based task called MATB-II. This task was designed to induce varying levels of mental workload by adjusting the complexity of different subtasks over time. During the experiment, participants reported their perceived cognitive load every 10 seconds using a self-report scale.
The dataset includes four modalities of physiological and gaze data:
- Electrocardiography (ECG) to measure heart activity
- Electrodermal Activity (EDA) to measure sweat levels
- Electroencephalogram (EEG) to measure brain waves
- Gaze tracking to measure eye movements
The researchers provide benchmark results using both machine learning and deep learning models for binary classification of high vs. low cognitive load. For 10-fold cross-validation, a convolutional neural network (CNN) model achieved the best performance using ECG, EDA, and gaze data. However, for the more challenging leave-one-subject-out (LOSO) evaluation, the best model used ECG, EDA, and EEG data.
Critical Analysis
The CLARE dataset and accompanying benchmarks provide a valuable resource for researchers studying cognitive load assessment using multimodal signals. The inclusion of self-reported cognitive load scores as ground truth labels is a key strength, as this allows for training and evaluating models that can accurately predict a person's subjective experience of mental effort.
However, the dataset is limited to 24 participants, which may constrain the ability to generalize the findings to a broader population. Additionally, the cognitive load task (MATB-II) was computer-based, raising questions about how well the results would translate to real-world scenarios with different types of mental workload.
Further research could explore the dataset's applicability in more diverse settings, as well as investigate the potential for continual learning approaches to improve the robustness and generalization of cognitive load assessment models.
Conclusion
The CLARE dataset represents an important step forward in the development of technologies for real-time cognitive load assessment. By providing a multimodal dataset with ground truth labels, the researchers have enabled the creation of more accurate and reliable models for detecting mental workload. This could have significant implications for applications ranging from operator performance monitoring to adaptive user interfaces, ultimately leading to systems that are better tailored to the cognitive needs of individual users.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
📊
0
CLARE: Cognitive Load Assessment in REaltime with Multimodal Data
Anubhav Bhatti, Prithila Angkan, Behnam Behinaein, Zunayed Mahmud, Dirk Rodenburg, Heather Braund, P. James Mclellan, Aaron Ruberto, Geoffery Harrison, Daryl Wilson, Adam Szulewski, Dan Howes, Ali Etemad, Paul Hungler
We present a novel multimodal dataset for Cognitive Load Assessment in REaltime (CLARE). The dataset contains physiological and gaze data from 24 participants with self-reported cognitive load scores as ground-truth labels. The dataset consists of four modalities, namely, Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking. To map diverse levels of mental load on participants during experiments, each participant completed four nine-minutes sessions on a computer-based operator performance and mental workload task (the MATB-II software) with varying levels of complexity in one minute segments. During the experiment, participants reported their cognitive load every 10 seconds. For the dataset, we also provide benchmark binary classification results with machine learning and deep learning models on two different evaluation schemes, namely, 10-fold and leave-one-subject-out (LOSO) cross-validation. Benchmark results show that for 10-fold evaluation, the convolutional neural network (CNN) based deep learning model achieves the best classification performance with ECG, EDA, and Gaze. In contrast, for LOSO, the best performance is achieved by the deep learning model with ECG, EDA, and EEG.
Read more4/29/2024
🔍
0
Wearable Device-Based Physiological Signal Monitoring: An Assessment Study of Cognitive Load Across Tasks
Ling He, Yanxin Chen, Wenqi Wang, Shuting He, Xiaoqiang Hu
This study employs cutting-edge wearable monitoring technology to conduct high-precision, high-temporal-resolution (1-second interval) cognitive load assessment on electroencephalogram (EEG) data from the FP1 channel and heart rate variability (HRV) data of secondary vocational students. By jointly analyzing these two critical physiological indicators, the research delves into their application value in assessing cognitive load among secondary vocational students and their utility across various tasks. The study designed two experiments to validate the efficacy of the proposed approach: Initially, a random forest classification model, developed using the N-BACK task, enabled the precise decoding of physiological signal characteristics in secondary vocational students under different levels of cognitive load, achieving a classification accuracy of 97%. Subsequently, this classification model was applied in a cross-task experiment involving the National Computer Rank Examination (Level-1), demonstrating the method's significant applicability and cross-task transferability in diverse learning contexts. Conducted with high portability, this research holds substantial theoretical and practical significance for optimizing teaching resource allocation in secondary vocational education, as well as for cognitive load assessment methods and monitoring. Currently, the research findings are undergoing trial implementation in the school.
Read more7/4/2024
0
CALM: Cognitive Assessment using Light-insensitive Model
Akhil Meethal, Anita Paas, Nerea Urrestilla Anguiozar, David St-Onge
The demand for cognitive load assessment with low-cost easy-to-use equipment is increasing, with applications ranging from safety-critical industries to entertainment. Though pupillometry is an attractive solution for cognitive load estimation in such applications, its sensitivity to light makes it less robust under varying lighting conditions. Multimodal data acquisition provides a viable alternative, where pupillometry is combined with electrocardiography (ECG) or electroencephalography (EEG). In this work, we study the sensitivity of pupillometry-based cognitive load estimation to light. By collecting heart rate variability (HRV) data during the same experimental sessions, we analyze how the multimodal data reduces this sensitivity and increases robustness to light conditions. In addition to this, we compared the performance in multimodal settings using the HRV data obtained from low-cost fitness-grade equipment to that from clinical-grade equipment by synchronously collecting data from both devices for all task conditions. Our results indicate that multimodal data improves the robustness of cognitive load estimation under changes in light conditions and improves the accuracy by more than 20% points over assessment based on pupillometry alone. In addition to that, the fitness grade device is observed to be a potential alternative to the clinical grade one, even in controlled laboratory settings.
Read more9/9/2024
📊
0
Predicting Cognitive Load Using Sensor Data in a Literacy Game
Minghao Cai, Carrie Demmans Epp
Educational games are being increasingly used to support self-paced learning. However, educators and system designers often face challenges in monitoring student affect and cognitive load. Existing assessments in game-based learning environments (GBLEs) tend to focus more on outcomes rather than processes, potentially overlooking key aspects of the learning journey that include learner affect and cognitive load. To address this issue, we collected data and trained a model to track learner cognitive load while they used an online literacy game for English. We collected affect-related physiological data and pupil data during gameplay to enable the development of models that identify these latent characteristics of learner processes. Our model indicates the feasibility of using these data to track cognitive load in GBLEs. Our multimodal model distinguished different levels of cognitive load, achieving the highest Kappa (.417) and accuracy (70%). Our model reveals the importance of including affect-related features (i.e., EDA and heart rate) when predicting cognitive load and extends recent findings suggesting the benefit of using multiple channels when modeling latent aspects of learner processes. Findings also suggest that cognitive load tracking could now be used to facilitate the creation of personalized learning experiences.
Read more5/10/2024