LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising
0
Sign in to get full access
Overview
- Introduces a large-scale, real-world paired dataset called LED (Low-light Event-based Denoising) for developing and evaluating event camera denoising algorithms.
- Event cameras are novel sensors that capture brightness changes in the scene, rather than traditional frame-based imaging, making them well-suited for low-light conditions.
- The LED dataset provides high-quality ground truth images paired with corresponding event data, addressing the lack of such datasets in the field.
Plain English Explanation
Event cameras are a type of sensor that work differently from traditional cameras. Instead of capturing full images at a fixed rate, they only record changes in brightness in the scene. This makes them particularly useful in low-light conditions, where regular cameras often struggle.
To help researchers develop better algorithms for cleaning up the noise in event camera data, the authors of this paper created a new dataset called LED (Low-light Event-based Denoising). The LED dataset contains many pairs of event data and high-quality reference images taken in the real world. This allows algorithms to be trained and tested on realistic low-light scenes.
Having this kind of paired dataset is important because it gives researchers the ground truth information they need to improve event camera denoising methods. Without it, it's much harder to know how well the algorithms are performing. The LED dataset aims to fill this gap and advance the state of the art in event camera technology.
Technical Explanation
The paper introduces the LED (Low-light Event-based Denoising) dataset, a large-scale, real-world paired dataset for developing and evaluating event camera denoising algorithms. Event cameras are novel sensors that capture brightness changes in a scene, rather than traditional frame-based imaging, making them well-suited for low-light conditions.
The LED dataset provides high-quality ground truth images paired with corresponding event data, addressing the lack of such datasets in the field. The authors describe the dataset collection process, which involved capturing synchronized event data and high-quality reference images under various low-light conditions using a specially designed rig. They also provide an analysis of the dataset characteristics and benchmark the performance of several state-of-the-art denoising methods on the LED dataset.
The availability of the LED dataset is expected to facilitate the development of more robust and effective event camera denoising algorithms, ultimately improving the capabilities of event cameras in low-light environments. The dataset is publicly available to the research community.
Critical Analysis
The LED dataset represents an important contribution to the field of event camera research, as it provides a much-needed resource for developing and evaluating denoising algorithms in realistic low-light conditions. The authors have carefully designed the data collection process to ensure high-quality ground truth images and event data, which is crucial for meaningful algorithm evaluation.
However, the paper does not address potential biases or limitations in the dataset, such as the restricted set of scenes and lighting conditions included. Additionally, the benchmark results presented provide a useful baseline, but more comprehensive testing on a wider range of algorithms and diverse real-world scenarios would be valuable.
Future research could explore ways to further expand the LED dataset, such as incorporating more diverse environments, dynamic scenes, and even challenging weather conditions. Investigating the generalization capabilities of denoising methods across different datasets would also be an interesting avenue for future work.
Conclusion
The LED (Low-light Event-based Denoising) dataset introduced in this paper represents a significant advancement in the field of event camera research. By providing a large-scale, real-world dataset of paired event data and high-quality reference images, the authors have created a valuable resource for the development and evaluation of event camera denoising algorithms.
The availability of this dataset is expected to accelerate progress in improving the performance of event cameras in low-light conditions, a crucial capability for many real-world applications. As event camera technology continues to evolve, the LED dataset will play an important role in driving further advancements and unlocking the full potential of these novel sensors.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising
Yuxing Duan, Shihan Peng, Lin Zhu, Wei Zhang, Yi Chang, Sheng Zhong, Luxin Yan
Event camera has significant advantages in capturing dynamic scene information while being prone to noise interference, particularly in challenging conditions like low threshold and low illumination. However, most existing research focuses on gentle situations, hindering event camera applications in realistic complex scenarios. To tackle this limitation and advance the field, we construct a new paired real-world event denoising dataset (LED), including 3K sequences with 18K seconds of high-resolution (1200*680) event streams and showing three notable distinctions compared to others: diverse noise levels and scenes, larger-scale with high-resolution, and high-quality GT. Specifically, it contains stepped parameters and varying illumination with diverse scenarios. Moreover, based on the property of noise events inconsistency and signal events consistency, we propose a novel effective denoising framework(DED) using homogeneous dual events to generate the GT with better separating noise from the raw. Furthermore, we design a bio-inspired baseline leveraging Leaky-Integrate-and-Fire (LIF) neurons with dynamic thresholds to realize accurate denoising. The experimental results demonstrate that the remarkable performance of the proposed approach on different datasets.The dataset and code are at https://github.com/Yee-Sing/led.
Read more5/31/2024
0
EvLight++: Low-Light Video Enhancement with an Event Camera: A Large-Scale Real-World Dataset, Novel Method, and More
Kanghao Chen, Guoqiang Liang, Hangyu Li, Yunfan Lu, Lin Wang
Event cameras offer significant advantages for low-light video enhancement, primarily due to their high dynamic range. Current research, however, is severely limited by the absence of large-scale, real-world, and spatio-temporally aligned event-video datasets. To address this, we introduce a large-scale dataset with over 30,000 pairs of frames and events captured under varying illumination. This dataset was curated using a robotic arm that traces a consistent non-linear trajectory, achieving spatial alignment precision under 0.03mm and temporal alignment with errors under 0.01s for 90% of the dataset. Based on the dataset, we propose textbf{EvLight++}, a novel event-guided low-light video enhancement approach designed for robust performance in real-world scenarios. Firstly, we design a multi-scale holistic fusion branch to integrate structural and textural information from both images and events. To counteract variations in regional illumination and noise, we introduce Signal-to-Noise Ratio (SNR)-guided regional feature selection, enhancing features from high SNR regions and augmenting those from low SNR regions by extracting structural information from events. To incorporate temporal information and ensure temporal coherence, we further introduce a recurrent module and temporal loss in the whole pipeline. Extensive experiments on our and the synthetic SDSD dataset demonstrate that EvLight++ significantly outperforms both single image- and video-based methods by 1.37 dB and 3.71 dB, respectively. To further explore its potential in downstream tasks like semantic segmentation and monocular depth estimation, we extend our datasets by adding pseudo segmentation and depth labels via meticulous annotation efforts with foundation models. Experiments under diverse low-light scenes show that the enhanced results achieve a 15.97% improvement in mIoU for semantic segmentation.
Read more8/30/2024
0
Towards Real-world Event-guided Low-light Video Enhancement and Deblurring
Taewoo Kim, Jaeseok Jeong, Hoonhee Cho, Yuhwan Jeong, Kuk-Jin Yoon
In low-light conditions, capturing videos with frame-based cameras often requires long exposure times, resulting in motion blur and reduced visibility. While frame-based motion deblurring and low-light enhancement have been studied, they still pose significant challenges. Event cameras have emerged as a promising solution for improving image quality in low-light environments and addressing motion blur. They provide two key advantages: capturing scene details well even in low light due to their high dynamic range, and effectively capturing motion information during long exposures due to their high temporal resolution. Despite efforts to tackle low-light enhancement and motion deblurring using event cameras separately, previous work has not addressed both simultaneously. To explore the joint task, we first establish real-world datasets for event-guided low-light enhancement and deblurring using a hybrid camera system based on beam splitters. Subsequently, we introduce an end-to-end framework to effectively handle these tasks. Our framework incorporates a module to efficiently leverage temporal information from events and frames. Furthermore, we propose a module to utilize cross-modal feature information to employ a low-pass filter for noise suppression while enhancing the main structural information. Our proposed method significantly outperforms existing approaches in addressing the joint task. Our project pages are available at https://github.com/intelpro/ELEDNet.
Read more8/28/2024
0
Seeing Motion at Nighttime with an Event Camera
Haoyue Liu, Shihan Peng, Lin Zhu, Yi Chang, Hanyu Zhou, Luxin Yan
We focus on a very challenging task: imaging at nighttime dynamic scenes. Most previous methods rely on the low-light enhancement of a conventional RGB camera. However, they would inevitably face a dilemma between the long exposure time of nighttime and the motion blur of dynamic scenes. Event cameras react to dynamic changes with higher temporal resolution (microsecond) and higher dynamic range (120dB), offering an alternative solution. In this work, we present a novel nighttime dynamic imaging method with an event camera. Specifically, we discover that the event at nighttime exhibits temporal trailing characteristics and spatial non-stationary distribution. Consequently, we propose a nighttime event reconstruction network (NER-Net) which mainly includes a learnable event timestamps calibration module (LETC) to align the temporal trailing events and a non-uniform illumination aware module (NIAM) to stabilize the spatiotemporal distribution of events. Moreover, we construct a paired real low-light event dataset (RLED) through a co-axial imaging system, including 64,200 spatially and temporally aligned image GTs and low-light events. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods in terms of visual quality and generalization ability on real-world nighttime datasets. The project are available at: https://github.com/Liu-haoyue/NER-Net.
Read more4/19/2024