Open Gaze: Open Source eye tracker for smartphone devices using Deep Learning
0
🤿
Sign in to get full access
Overview
- Smartphone-based eye tracking has significant potential to expand vision research by enabling large-scale participation.
- Prior studies have focused on desktop displays using specialized, costly hardware, with limited insight into smartphone-based ocular movement patterns.
- This paper presents an open-source, smartphone-based gaze tracker that aims to match the accuracy of a proprietary Google solution.
Plain English Explanation
Eye tracking technology has been widely used in various fields, such as vision research, language analysis, and usability assessment. However, most of these studies have been conducted using large desktop displays and specialized, expensive eye tracking hardware, which limits their scalability.
Surprisingly, there has been little research exploring how people's eyes move on smartphones, even though smartphones are now ubiquitous and widely used. To address this gap, the researchers in this paper have developed an open-source, smartphone-based eye tracking system. Their goal is to create a solution that can accurately track eye movements on smartphones, without the need for additional hardware.
By leveraging machine learning techniques, the researchers have developed a smartphone-based eye tracking solution that can match the accuracy of state-of-the-art mobile eye trackers, which are typically much more expensive. This new approach allows for large-scale participation in vision research, as it can be easily deployed on smartphones that people already own.
The researchers used a dataset called the MIT GazeCapture dataset, which is available to researchers, to replicate findings from previous studies on how people's eyes move during various tasks, such as reading and viewing natural images. This demonstrates the potential of smartphone-based eye tracking to advance vision research, as well as applications in areas like accessibility enhancement and healthcare.
Technical Explanation
The researchers present an open-source implementation of a smartphone-based gaze tracker that emulates the methodology proposed by a proprietary Google solution. Their focus is on achieving accuracy comparable to the Google approach, without the need for additional hardware.
By integrating machine learning techniques, the researchers unveil an accurate eye tracking solution that is native to smartphones. This approach demonstrates precision akin to state-of-the-art mobile eye trackers, which are significantly more expensive.
Leveraging the MIT GazeCapture dataset, the researchers successfully replicate findings from previous studies on ocular motion behavior in oculomotor tasks and saliency analyses during natural image observation. Additionally, they explore the application of smartphone-based gaze tracking in discerning reading comprehension challenges.
The researchers emphasize the scalability of this smartphone-based gaze tracking approach, which can accommodate participation from thousands of subjects with explicit consent. This scalability not only fosters advancements in vision research, but also extends the benefits to domains such as accessibility enhancement and healthcare applications.
Critical Analysis
The researchers acknowledge the limitations of their approach, such as the need for explicit consent from participants and the potential bias in the MIT GazeCapture dataset, which was collected from a specific demographic. They also note that further research is required to address the impact of variable lighting conditions and screen sizes on the accuracy of the eye tracking system.
Additionally, while the researchers have demonstrated the potential of smartphone-based eye tracking, the long-term privacy implications of widespread data collection on personal devices should be carefully considered. Ethical concerns, such as data privacy and the potential for misuse, need to be thoroughly addressed to ensure the responsible development and deployment of this technology.
Conclusion
This research presents a promising step towards democratizing eye tracking technology by leveraging the ubiquity of smartphones. The open-source, smartphone-based gaze tracker developed by the researchers has the potential to significantly expand vision research by enabling large-scale participation, while also offering applications in accessibility enhancement and healthcare.
However, the researchers acknowledge the need to address the limitations and ethical concerns associated with this technology. Ongoing efforts to improve accuracy, privacy, and user consent will be crucial in realizing the full potential of smartphone-based eye tracking and ensuring its responsible adoption across various domains.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🤿
0
Open Gaze: Open Source eye tracker for smartphone devices using Deep Learning
Sushmanth reddy, Jyothi Swaroop Reddy
Eye tracking has been a pivotal tool in diverse fields such as vision research, language analysis, and usability assessment. The majority of prior investigations, however, have concentrated on expansive desktop displays employing specialized, costly eye tracking hardware that lacks scalability. Remarkably little insight exists into ocular movement patterns on smartphones, despite their widespread adoption and significant usage. In this manuscript, we present an open-source implementation of a smartphone-based gaze tracker that emulates the methodology proposed by a GooglePaper (whose source code remains proprietary). Our focus is on attaining accuracy comparable to that attained through the GooglePaper's methodology, without the necessity for supplementary hardware. Through the integration of machine learning techniques, we unveil an accurate eye tracking solution that is native to smartphones. Our approach demonstrates precision akin to the state-of-the-art mobile eye trackers, which are characterized by a cost that is two orders of magnitude higher. Leveraging the vast MIT GazeCapture dataset, which is available through registration on the dataset's website, we successfully replicate crucial findings from previous studies concerning ocular motion behavior in oculomotor tasks and saliency analyses during natural image observation. Furthermore, we emphasize the applicability of smartphone-based gaze tracking in discerning reading comprehension challenges. Our findings exhibit the inherent potential to amplify eye movement research by significant proportions, accommodating participation from thousands of subjects with explicit consent. This scalability not only fosters advancements in vision research, but also extends its benefits to domains such as accessibility enhancement and healthcare applications.
Read more9/5/2024
0
Smartphone-based Eye Tracking System using Edge Intelligence and Model Optimisation
Nishan Gunawardena, Gough Yumu Lui, Jeewani Anupama Ginige, Bahman Javadi
A significant limitation of current smartphone-based eye-tracking algorithms is their low accuracy when applied to video-type visual stimuli, as they are typically trained on static images. Also, the increasing demand for real-time interactive applications like games, VR, and AR on smartphones requires overcoming the limitations posed by resource constraints such as limited computational power, battery life, and network bandwidth. Therefore, we developed two new smartphone eye-tracking techniques for video-type visuals by combining Convolutional Neural Networks (CNN) with two different Recurrent Neural Networks (RNN), namely Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). Our CNN+LSTM and CNN+GRU models achieved an average Root Mean Square Error of 0.955cm and 1.091cm, respectively. To address the computational constraints of smartphones, we developed an edge intelligence architecture to enhance the performance of smartphone-based eye tracking. We applied various optimisation methods like quantisation and pruning to deep learning models for better energy, CPU, and memory usage on edge devices, focusing on real-time processing. Using model quantisation, the model inference time in the CNN+LSTM and CNN+GRU models was reduced by 21.72% and 19.50%, respectively, on edge devices.
Read more8/23/2024
0
3D Gaze Tracking for Studying Collaborative Interactions in Mixed-Reality Environments
Eduardo Davalos, Yike Zhang, Ashwin T. S., Joyce H. Fonteles, Umesh Timalsina, Guatam Biswas
This study presents a novel framework for 3D gaze tracking tailored for mixed-reality settings, aimed at enhancing joint attention and collaborative efforts in team-based scenarios. Conventional gaze tracking, often limited by monocular cameras and traditional eye-tracking apparatus, struggles with simultaneous data synchronization and analysis from multiple participants in group contexts. Our proposed framework leverages state-of-the-art computer vision and machine learning techniques to overcome these obstacles, enabling precise 3D gaze estimation without dependence on specialized hardware or complex data fusion. Utilizing facial recognition and deep learning, the framework achieves real-time, tracking of gaze patterns across several individuals, addressing common depth estimation errors, and ensuring spatial and identity consistency within the dataset. Empirical results demonstrate the accuracy and reliability of our method in group environments. This provides mechanisms for significant advances in behavior and interaction analysis in educational and professional training applications in dynamic and unstructured environments.
Read more6/18/2024
0
Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality
Kevin Barkevich, Reynold Bailey, Gabriel J. Diaz
Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.
Read more4/1/2024