Event-Based Eye Tracking. AIS 2024 Challenge Survey
0
Sign in to get full access
Overview
- This paper surveys the 3ET+ dataset and the Event-Based Eye Tracking (EBTE) challenge, which aims to advance research in event-based eye tracking.
- The 3ET+ dataset provides high-resolution, high-speed eye tracking data captured using event-based sensors, presenting new opportunities for developing more accurate and efficient eye tracking models.
- The EBTE challenge calls for participants to develop novel techniques for processing this unique event-based eye tracking data, with the goal of driving progress in this emerging field.
Plain English Explanation
The paper discusses a new dataset and challenge related to a novel approach for eye tracking called "event-based eye tracking." Event-based eye tracking is a technique that uses specialized sensors to capture changes in eye movement and gaze rather than relying on traditional frame-based video.
The 3ET+ dataset provides high-quality, high-speed eye tracking data collected using these event-based sensors. This data offers new opportunities for developing more accurate and efficient eye tracking models compared to traditional approaches.
The paper also introduces the Event-Based Eye Tracking (EBTE) challenge, which calls on researchers to develop innovative techniques for processing and analyzing this unique event-based eye tracking data. The goal is to advance the state-of-the-art in event-based eye tracking, which could lead to improvements in a variety of applications, such as human-computer interaction, virtual/augmented reality, and assistive technologies.
Technical Explanation
The paper presents the 3ET+ dataset, which was collected using event-based eye tracking sensors. These sensors capture changes in eye movement and gaze rather than recording a continuous video stream like traditional eye trackers. This event-based approach provides high-resolution, high-speed data that could enable the development of more accurate and efficient eye tracking models.
The 3ET+ dataset includes eye tracking data from 32 participants performing various tasks, such as reading, scene viewing, and visual search. The dataset provides ground truth labels for gaze position, eye movements, and other eye-related metrics. Researchers can use this data to train and evaluate novel eye tracking algorithms that can effectively process the event-based eye tracking data.
The paper also introduces the Event-Based Eye Tracking (EBTE) challenge, which invites participants to develop advanced techniques for analyzing the 3ET+ dataset. The challenge aims to drive progress in this emerging field of event-based eye tracking, with potential applications in areas like human-computer interaction, virtual/augmented reality, and assistive technologies.
Critical Analysis
The paper provides a valuable resource for the research community by introducing the 3ET+ dataset and the EBTE challenge. The event-based eye tracking data offered by 3ET+ presents new opportunities for developing more accurate and efficient eye tracking models, but the paper acknowledges that processing this type of data requires novel algorithms and techniques.
One potential limitation of the 3ET+ dataset is the relatively small number of participants (32). While this is a common sample size in eye tracking studies, expanding the dataset to include a more diverse population could enhance the generalizability of any models developed using the data.
Additionally, the paper does not provide much insight into the specific challenges or difficulties associated with event-based eye tracking. Further research may be needed to understand the unique characteristics and requirements of this emerging technology, as well as any potential limitations or drawbacks compared to traditional frame-based eye tracking approaches.
Conclusion
The 3ET+ dataset and the EBTE challenge represent an important step forward in the field of event-based eye tracking. By providing high-quality, high-speed eye tracking data collected using specialized sensors, the 3ET+ dataset opens the door for researchers to develop innovative techniques that could lead to more accurate and efficient eye tracking models.
The EBTE challenge encourages the research community to push the boundaries of event-based eye tracking, with the potential to drive advancements in a variety of applications, such as human-computer interaction, virtual/augmented reality, and assistive technologies. As this field continues to evolve, the insights and solutions generated through the EBTE challenge could have far-reaching implications for how we understand and interact with the world around us.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Event-Based Eye Tracking. AIS 2024 Challenge Survey
Zuowen Wang, Chang Gao, Zongwei Wu, Marcos V. Conde, Radu Timofte, Shih-Chii Liu, Qinyu Chen, Zheng-jun Zha, Wei Zhai, Han Han, Bohao Liao, Yuliang Wu, Zengyu Wan, Zhong Wang, Yang Cao, Ganchao Tan, Jinze Chen, Yan Ru Pei, Sasskia Bruers, S'ebastien Crouzet, Douglas McLelland, Oliver Coenen, Baoheng Zhang, Yizhao Gao, Jingyuan Li, Hayden Kwok-Hay So, Philippe Bich, Chiara Boretti, Luciano Prono, Mircea Licu{a}, David Dinucu-Jianu, Cu{a}tu{a}lin Gr^iu, Xiaopeng Lin, Hongwei Ren, Bojun Cheng, Xinan Zhang, Valentin Vial, Anthony Yezzi, James Tsai
This survey reviews the AIS 2024 Event-Based Eye Tracking (EET) Challenge. The task of the challenge focuses on processing eye movement recorded with event cameras and predicting the pupil center of the eye. The challenge emphasizes efficient eye tracking with event cameras to achieve good task accuracy and efficiency trade-off. During the challenge period, 38 participants registered for the Kaggle competition, and 8 teams submitted a challenge factsheet. The novel and diverse methods from the submitted factsheets are reviewed and analyzed in this survey to advance future event-based eye tracking research.
Read more4/19/2024
0
Evaluating Image-Based Face and Eye Tracking with Event Cameras
Khadija Iddrisu, Waseem Shariff, Noel E. OConnor, Joseph Lemley, Suzanne Little
Event Cameras, also known as Neuromorphic sensors, capture changes in local light intensity at the pixel level, producing asynchronously generated data termed ``events''. This distinct data format mitigates common issues observed in conventional cameras, like under-sampling when capturing fast-moving objects, thereby preserving critical information that might otherwise be lost. However, leveraging this data often necessitates the development of specialized, handcrafted event representations that can integrate seamlessly with conventional Convolutional Neural Networks (CNNs), considering the unique attributes of event data. In this study, We evaluate event-based Face and Eye tracking. The core objective of our study is to showcase the viability of integrating conventional algorithms with event-based data, transformed into a frame format while preserving the unique benefits of event cameras. To validate our approach, we constructed a frame-based event dataset by simulating events between RGB frames derived from the publicly accessible Helen Dataset. We assess its utility for face and eye detection tasks through the application of GR-YOLO -- a pioneering technique derived from YOLOv3. This evaluation includes a comparative analysis with results derived from training the dataset with YOLOv8. Subsequently, the trained models were tested on real event streams from various iterations of Prophesee's event cameras and further evaluated on the Faces in Event Stream (FES) benchmark dataset. The models trained on our dataset shows a good prediction performance across all the datasets obtained for validation with the best results of a mean Average precision score of 0.91. Additionally, The models trained demonstrated robust performance on real event camera data under varying light conditions.
Read more8/21/2024
0
Co-designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNN
Baoheng Zhang, Yizhao Gao, Jingyuan Li, Hayden Kwok-Hay So
Eye-tracking technology is integral to numerous consumer electronics applications, particularly in the realm of virtual and augmented reality (VR/AR). These applications demand solutions that excel in three crucial aspects: low-latency, low-power consumption, and precision. Yet, achieving optimal performance across all these fronts presents a formidable challenge, necessitating a balance between sophisticated algorithms and efficient backend hardware implementations. In this study, we tackle this challenge through a synergistic software/hardware co-design of the system with an event camera. Leveraging the inherent sparsity of event-based input data, we integrate a novel sparse FPGA dataflow accelerator customized for submanifold sparse convolution neural networks (SCNN). The SCNN implemented on the accelerator can efficiently extract the embedding feature vector from each representation of event slices by only processing the non-zero activations. Subsequently, these vectors undergo further processing by a gated recurrent unit (GRU) and a fully connected layer on the host CPU to generate the eye centers. Deployment and evaluation of our system reveal outstanding performance metrics. On the Event-based Eye-Tracking-AIS2024 dataset, our system achieves 81% p5 accuracy, 99.5% p10 accuracy, and 3.71 Mean Euclidean Distance with 0.7 ms latency while only consuming 2.29 mJ per inference. Notably, our solution opens up opportunities for future eye-tracking systems. Code is available at https://github.com/CASR-HKU/ESDA/tree/eye_tracking.
Read more4/23/2024
0
A Lightweight Spatiotemporal Network for Online Eye Tracking with Event Camera
Yan Ru Pei, Sasskia Bruers, S'ebastien Crouzet, Douglas McLelland, Olivier Coenen
Event-based data are commonly encountered in edge computing environments where efficiency and low latency are critical. To interface with such data and leverage their rich temporal features, we propose a causal spatiotemporal convolutional network. This solution targets efficient implementation on edge-appropriate hardware with limited resources in three ways: 1) deliberately targets a simple architecture and set of operations (convolutions, ReLU activations) 2) can be configured to perform online inference efficiently via buffering of layer outputs 3) can achieve more than 90% activation sparsity through regularization during training, enabling very significant efficiency gains on event-based processors. In addition, we propose a general affine augmentation strategy acting directly on the events, which alleviates the problem of dataset scarcity for event-based systems. We apply our model on the AIS 2024 event-based eye tracking challenge, reaching a score of 0.9916 p10 accuracy on the Kaggle private testset.
Read more4/16/2024