fNIRS Analysis of Interaction Techniques in Touchscreen-Based Educational Gaming
0
Sign in to get full access
Overview
- This paper investigates the use of functional near-infrared spectroscopy (fNIRS) to analyze the cognitive processes involved in different touch-based interaction techniques used in educational gaming.
- The researchers compared user performance and brain activity patterns across three common touch-based interaction methods: direct touch, indirect touch, and touch with visual feedback.
- The goal was to provide insights into the optimal design of touch-based educational games to maximize engagement and learning.
Plain English Explanation
The paper looks at how people's brains respond to different ways of interacting with touchscreen games used for education. The researchers used a technology called fNIRS to measure brain activity while people played games that involved directly touching the screen, indirectly touching it with a pointer, or touching it while getting visual feedback.
The goal was to figure out which interaction method works best for keeping people engaged and helping them learn. Direct touch, indirect touch, and touch with visual feedback all have different pros and cons, and the researchers wanted to see how our brains respond to each one. This can help designers create more effective educational games that are easy and intuitive to use.
Technical Explanation
The researchers conducted an experiment where participants played three different touchscreen-based educational games, each using a different interaction technique: direct touch, indirect touch with a pointer, and touch with visual feedback.
While the participants played the games, the researchers used fNIRS to measure activity in their prefrontal cortex, which is involved in cognitive processing and decision-making. The researchers analyzed the brain activity patterns and performance data to understand the cognitive demands and engagement levels associated with each interaction method.
The results showed that the direct touch interaction led to the highest engagement and learning performance, as evidenced by greater brain activity in the prefrontal cortex. The indirect touch and touch with visual feedback conditions resulted in lower engagement and more cognitive effort.
Critical Analysis
The paper provides valuable insights into the cognitive impacts of different touch-based interaction techniques in educational gaming. However, the authors acknowledge several limitations:
- The study only examined brain activity in the prefrontal cortex, and may have missed activity in other brain regions involved in the interaction process.
- The experiments were conducted in a controlled laboratory setting, which may not fully reflect real-world gaming scenarios.
- The sample size was relatively small, and the results may not generalize to a broader population.
Additionally, the paper does not address potential individual differences in user preferences or prior experience with touchscreen technologies. Further research could explore how factors like age, gaming experience, or learning style might influence the optimal interaction technique for educational gaming.
Conclusion
This study suggests that direct touch interaction is the most effective approach for engaging users and supporting learning in touchscreen-based educational games. The fNIRS data indicates that this method requires the least cognitive effort while eliciting the strongest engagement-related brain activity.
These findings have important implications for the design of educational games and interactive learning tools. By understanding the cognitive mechanisms underlying different touch-based interaction techniques, developers can create more intuitive and effective educational experiences that maximize user engagement and learning outcomes.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
fNIRS Analysis of Interaction Techniques in Touchscreen-Based Educational Gaming
Shayla Sharmin, Elham Bakhshipour, Behdokht Kiafar, Md Fahim Abrar, Pinar Kullu, Nancy Getchell, Roghayeh Leila Barmaki
Touchscreens are becoming increasingly widespread in educational games, enhancing the quality of learner experience. Traditional metrics are often used to evaluate various input modalities, including hand and stylus. However, there exists a gap in understanding the cognitive impacts of these modalities during educational gameplay, which can be addressed through brain signal analysis to gain deeper insights into the underlying cognitive function and necessary brain resources for each condition. This facilitates a more precise comparison between conditions. In this study, we compared the brain signal and user experience of using hands and stylus on touchscreens while playing an educational game by analyzing hemodynamic response and self-reported measures. Participants engaged in a Unity-based educational quiz game using both hand and stylus on a touchscreen in a counterbalanced within-subject design. Oxygenated and deoxygenated hemoglobin data were collected using fNIRS, alongside quiz performance scores and standardized and customized user experience questionnaire ratings. Our findings show almost the same performance level with both input modalities, however, the hand requires less oxygen flow which suggests a lower cognitive effort than using a stylus while playing the educational game. Although the result shows that the stylus condition required more neural involvement than the hand condition, there is no significant difference between the use of both input modalities. However, there is a statistically significant difference in self-reported measures that support the findings mentioned above, favoring the hand that enhances understanding of modality effects in interactive educational environments.
Read more5/16/2024
0
Evaluating Force-based Haptics for Immersive Tangible Interactions with Surface Visualizations
Hamza Afzaal, Usman Alim
Haptic feedback provides an essential sensory stimulus crucial for interaction and analyzing three-dimensional spatio-temporal phenomena on surface visualizations. Given its ability to provide enhanced spatial perception and scene maneuverability, virtual reality (VR) catalyzes haptic interactions on surface visualizations. Various interaction modes, encompassing both mid-air and on-surface interactions -- with or without the application of assisting force stimuli -- have been explored using haptic force feedback devices. In this paper, we evaluate the use of on-surface and assisted on-surface haptic modes of interaction compared to a no-haptic interaction mode. A force-based haptic stylus is used for all three modalities; the on-surface mode uses collision based forces, whereas the assisted on-surface mode is accompanied by an additional snapping force. We conducted a within-subjects user study involving fundamental interaction tasks performed on surface visualizations. Keeping a consistent visual design across all three modes, our study incorporates tasks that require the localization of the highest, lowest, and random points on surfaces; and tasks that focus on brushing curves on surfaces with varying complexity and occlusion levels. Our findings show that participants took almost the same time to brush curves using all the interaction modes. They could draw smoother curves using the on-surface interaction modes compared to the no-haptic mode. However, the assisted on-surface mode provided better accuracy than the on-surface mode. The on-surface mode was slower in point localization, but the accuracy depended on the visual cues and occlusions associated with the tasks. Finally, we discuss participant feedback on using haptic force feedback as a tangible input modality and share takeaways to aid the design of haptics-based tangible interactions for surface visualizations.
Read more9/11/2024
0
Advancements in Tactile Hand Gesture Recognition for Enhanced Human-Machine Interaction
Chiara Fumelli, Anirvan Dutta, Mohsen Kaboli
Motivated by the growing interest in enhancing intuitive physical Human-Machine Interaction (HRI/HVI), this study aims to propose a robust tactile hand gesture recognition system. We performed a comprehensive evaluation of different hand gesture recognition approaches for a large area tactile sensing interface (touch interface) constructed from conductive textiles. Our evaluation encompassed traditional feature engineering methods, as well as contemporary deep learning techniques capable of real-time interpretation of a range of hand gestures, accommodating variations in hand sizes, movement velocities, applied pressure levels, and interaction points. Our extensive analysis of the various methods makes a significant contribution to tactile-based gesture recognition in the field of human-machine interaction.
Read more5/28/2024
0
HuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems
Sonia Castelo, Joao Rulff, Parikshit Solunke, Erin McGowan, Guande Wu, Iran Roman, Roque Lopez, Bea Steers, Qi Sun, Juan Bello, Bradley Feest, Michael Middleton, Ryan Mckendrick, Claudio Silva
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.
Read more7/18/2024