HuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems
0
Sign in to get full access
Overview
- This paper presents a visual analytics tool called "\systemname" that explores human behavior in augmented reality (AR) guidance systems using functional near-infrared spectroscopy (fNIRS) data.
- The tool provides interactive visualizations to analyze brain activity and task performance during AR-based interactions.
- The goal is to gain insights into how users engage with AR systems and how their cognitive and behavioral patterns change over time.
Plain English Explanation
The researchers have developed a new software tool called "\systemname" that helps analyze how people's brains and behavior change when they use augmented reality (AR) systems. AR is a technology that overlays digital information, like 3D models or instructions, on top of the real world that you see through a device like a smartphone or headset.
The tool uses a brain imaging technique called functional near-infrared spectroscopy (fNIRS) to measure activity in different parts of the brain as people interact with the AR system. It then provides interactive visualizations that let researchers and designers explore how the brain and behavior change over time during these AR interactions.
For example, the tool could show which parts of the brain "light up" when someone is trying to follow instructions displayed in an AR headset. Or it could reveal patterns in how people's focus and performance change as they get more experienced with the AR system. This kind of detailed insight into the human side of AR can help improve the design and effectiveness of these emerging technologies.
Technical Explanation
The paper introduces "\systemname", a visual analytics tool designed to explore human behavior and brain activity during interactions with AR guidance systems. The tool leverages functional near-infrared spectroscopy (fNIRS) - a non-invasive neuroimaging technique that measures changes in oxygenation levels in the brain - to capture users' cognitive and attentional states.
The system provides interactive visualizations that allow researchers to analyze patterns in users' task performance, gaze behavior, and brain activity over time. This includes visualizations of fNIRS data to reveal insights about the cognitive processes underlying AR-based interactions.
The researchers demonstrate the tool's capabilities through a user study involving an AR-based assembly task. The study explores how users' brain activity, focus, and performance evolve as they become more experienced with the AR guidance system. The results provide insights into the cognitive demands of AR interactions and inform the design of more effective AR interfaces.
Critical Analysis
The paper presents a novel and promising approach to understanding human behavior in AR systems. The use of fNIRS to capture real-time brain activity provides a valuable window into the cognitive processes involved in AR interactions. The interactive visualizations enable in-depth analysis of how users engage with these systems over time.
However, the study has some limitations. The sample size was relatively small, and the task was relatively simple. More research is needed to explore the generalizability of the findings to more complex AR applications and diverse user populations.
Additionally, the paper does not address potential privacy and ethical concerns around the use of neuroimaging data in user experience research. As these technologies become more advanced, it will be important to carefully consider the implications and develop appropriate safeguards.
Overall, the \systemname tool represents an important step towards advancing the use of neuroimaging in HCI research. By providing a deeper understanding of the human factors involved in AR interactions, it can inform the design of more intuitive and user-friendly AR systems.
Conclusion
The \systemname tool offers a novel approach to studying human behavior in AR guidance systems using fNIRS neuroimaging data. The interactive visualizations provide valuable insights into the cognitive processes and performance patterns involved in AR interactions, which can inform the design of more effective and user-friendly AR interfaces.
While the study has some limitations, the research represents an important step forward in using neuroimaging techniques to understand the human side of emerging technologies like AR. As these technologies continue to evolve, tools like \systemname will become increasingly valuable for optimizing user experiences and uncovering the cognitive factors that drive human-AR interactions.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
HuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems
Sonia Castelo, Joao Rulff, Parikshit Solunke, Erin McGowan, Guande Wu, Iran Roman, Roque Lopez, Bea Steers, Qi Sun, Juan Bello, Bradley Feest, Michael Middleton, Ryan Mckendrick, Claudio Silva
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.
Read more7/18/2024
🗣️
0
Adaptive Human-Swarm Interaction based on Workload Measurement using Functional Near-Infrared Spectroscopy
Ayodeji O. Abioye, Aleksandra Landowska, William Hunt, Horia Maior, Sarvapali D. Ramchurn, Mohammad Naiseh, Alec Banks, Mohammad D. Soorati
One of the challenges of human-swarm interaction (HSI) is how to manage the operator's workload. In order to do this, we propose a novel neurofeedback technique for the real-time measurement of workload using functional near-infrared spectroscopy (fNIRS). The objective is to develop a baseline for workload measurement in human-swarm interaction using fNIRS and to develop an interface that dynamically adapts to the operator's workload. The proposed method consists of using fNIRS device to measure brain activity, process this through a machine learning algorithm, and pass it on to the HSI interface. By dynamically adapting the HSI interface, the swarm operator's workload could be reduced and the performance improved.
Read more5/14/2024
2
Progress Towards Decoding Visual Imagery via fNIRS
Michel Adamic, Wellington Avelino, Anna Brandenberger, Bryan Chiang, Hunter Davis, Stephen Fay, Andrew Gregory, Aayush Gupta, Raphael Hotter, Grace Jiang, Fiona Leng, Stephen Polcyn, Thomas Ribeiro, Paul Scotti, Michelle Wang, Marley Xiong, Jonathan Xu
We demonstrate the possibility of reconstructing images from fNIRS brain activity and start building a prototype to match the required specs. By training an image reconstruction model on downsampled fMRI data, we discovered that cm-scale spatial resolution is sufficient for image generation. We obtained 71% retrieval accuracy with 1-cm resolution, compared to 93% on the full-resolution fMRI, and 20% with 2-cm resolution. With simulations and high-density tomography, we found that time-domain fNIRS can achieve 1-cm resolution, compared to 2-cm resolution for continuous-wave fNIRS. Lastly, we share designs for a prototype time-domain fNIRS device, consisting of a laser driver, a single photon detector, and a time-to-digital converter system.
Read more6/26/2024
🏷️
0
Functional near-infrared spectroscopy (fNIRS) and Eye tracking for Cognitive Load classification in a Driving Simulator Using Deep Learning
Mehshan Ahmed Khan, Houshyar Asadi, Mohammad Reza Chalak Qazani, Chee Peng Lim, Saied Nahavandi
Motion simulators allow researchers to safely investigate the interaction of drivers with a vehicle. However, many studies that use driving simulator data to predict cognitive load only employ two levels of workload, leaving a gap in research on employing deep learning methodologies to analyze cognitive load, especially in challenging low-light conditions. Often, studies overlook or solely focus on scenarios in bright daylight. To address this gap and understand the correlation between performance and cognitive load, this study employs functional near-infrared spectroscopy (fNIRS) and eye-tracking data, including fixation duration and gaze direction, during simulated driving tasks in low visibility conditions, inducing various mental workloads. The first stage involves the statistical estimation of useful features from fNIRS and eye-tracking data. ANOVA will be applied to the signals to identify significant channels from fNIRS signals. Optimal features from fNIRS, eye-tracking and vehicle dynamics are then combined in one chunk as input to the CNN and LSTM model to predict workload variations. The proposed CNN-LSTM model achieved 99% accuracy with neurological data and 89% with vehicle dynamics to predict cognitive load, indicating potential for real-time assessment of driver mental state and guide designers for the development of safe adaptive systems.
Read more8/14/2024