FastHDRNet: A new efficient method for SDR-to-HDR Translation
0
🏋️
Sign in to get full access
Overview
- Modern displays can now show video with high dynamic range (HDR) and a wide color gamut (WCG).
- However, most available video content is still in standard dynamic range (SDR).
- Existing deep neural network (DNN) based SDR to HDR conversion methods outperform traditional methods, but they are either too large to implement or produce poor visual quality.
Plain English Explanation
The paper proposes a new neural network called FastHDRNet that can efficiently convert SDR video to HDR video. This is an important problem because while modern displays can show HDR and wide color content, most existing video is still in the older SDR format.
The key idea is to use a lightweight neural network with two parts: an "Adaptive Universal Color Transformation" that adjusts the overall color and brightness, and a "Local Enhancement" part that fine-tunes the details. This allows the network to efficiently process the video while maintaining high visual quality, without the drawbacks of prior DNN-based methods.
Technical Explanation
The proposed FastHDRNet has two main components:
-
Adaptive Universal Color Transformation: This part of the network adjusts the overall color and brightness of the SDR input to match the characteristics of HDR video.
-
Local Enhancement: This component then refines the details and local contrast of the image to further improve the HDR appearance.
The architecture is designed to be lightweight and efficient, utilizing global statistics and local information. Experiments show that FastHDRNet achieves state-of-the-art performance in both quantitative metrics and visual quality, while also having a fast inference speed.
Critical Analysis
The paper demonstrates impressive results, but it would be helpful to know more about the limitations or potential issues with the approach. For example, how does the network handle challenging lighting conditions or specific types of content that may be difficult to convert effectively?
Additionally, while the authors claim the network is lightweight, it would be useful to have more details on the computational requirements and memory usage compared to prior methods. This could help assess the real-world feasibility and scalability of deploying FastHDRNet.
Conclusion
This research proposes a novel neural network called FastHDRNet that can efficiently convert standard dynamic range (SDR) video to high dynamic range (HDR) video. By using a lightweight architecture with global and local processing components, the method achieves state-of-the-art performance while maintaining fast inference speeds.
This work addresses an important problem in the field of video display technology, as modern displays are capable of showing HDR and wide color content, but most existing video content is still in the older SDR format. The FastHDRNet approach could help bridge this gap and enable more widespread adoption of HDR video.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🏋️
0
FastHDRNet: A new efficient method for SDR-to-HDR Translation
Siyuan Tian, Hao Wang, Yiren Rong, Junhao Wang, Renjie Dai, Zhengxiao He
Modern displays nowadays possess the capability to render video content with a high dynamic range (HDR) and an extensive color gamut .However, the majority of available resources are still in standard dynamic range (SDR). Therefore, we need to identify an effective methodology for this objective.The existing deep neural networks (DNN) based SDR to HDR conversion methods outperforms conventional methods, but they are either too large to implement or generate some terrible artifacts. We propose a neural network for SDR to HDR conversion, termed FastHDRNet. This network includes two parts, Adaptive Universal Color Transformation (AUCT) and Local Enhancement (LE). The architecture is designed as a lightweight network that utilizes global statistics and local information with super high efficiency. After the experiment, we find that our proposed method achieves state-of-the-art performance in both quantitative comparisons and visual quality with a lightweight structure and a enhanced infer speed.
Read more5/14/2024
0
Efficient HDR Reconstruction from Real-World Raw Images
Qirui Yang, Yihao Liu, Qihua Chen, Huanjing Yue, Kun Li, Jingyu Yang
The widespread usage of high-definition screens on edge devices stimulates a strong demand for efficient high dynamic range (HDR) algorithms. However, many existing HDR methods either deliver unsatisfactory results or consume too much computational and memory resources, hindering their application to high-resolution images (usually with more than 12 megapixels) in practice. In addition, existing HDR dataset collection methods often are labor-intensive. In this work, in a new aspect, we discover an excellent opportunity for HDR reconstructing directly from raw images and investigating novel neural network structures that benefit the deployment of mobile devices. Our key insights are threefold: (1) we develop a lightweight-efficient HDR model, RepUNet, using the structural re-parameterization technique to achieve fast and robust HDR; (2) we design a new computational raw HDR data formation pipeline and construct a real-world raw HDR dataset, RealRaw-HDR; (3) we propose a plug-and-play motion alignment loss to mitigate motion ghosting under limited bandwidth conditions. Our model contains less than 830K parameters and takes less than 3 ms to process an image of 4K resolution using one RTX 3090 GPU. While being highly efficient, our model also outperforms the state-of-the-art HDR methods in terms of PSNR, SSIM, and a color difference metric.
Read more6/6/2024
0
Adapting Pretrained Networks for Image Quality Assessment on High Dynamic Range Displays
Andrei Chubarau, Hyunjin Yoo, Tara Akhavan, James Clark
Conventional image quality metrics (IQMs), such as PSNR and SSIM, are designed for perceptually uniform gamma-encoded pixel values and cannot be directly applied to perceptually non-uniform linear high-dynamic-range (HDR) colors. Similarly, most of the available datasets consist of standard-dynamic-range (SDR) images collected in standard and possibly uncontrolled viewing conditions. Popular pre-trained neural networks are likewise intended for SDR inputs, restricting their direct application to HDR content. On the other hand, training HDR models from scratch is challenging due to limited available HDR data. In this work, we explore more effective approaches for training deep learning-based models for image quality assessment (IQA) on HDR data. We leverage networks pre-trained on SDR data (source domain) and re-target these models to HDR (target domain) with additional fine-tuning and domain adaptation. We validate our methods on the available HDR IQA datasets, demonstrating that models trained with our combined recipe outperform previous baselines, converge much quicker, and reliably generalize to HDR inputs.
Read more5/2/2024
0
HDRT: Infrared Capture for HDR Imaging
Jingchao Peng, Thomas Bashford-Rogers, Francesco Banterle, Haitao Zhao, Kurt Debattista
Capturing real world lighting is a long standing challenge in imaging and most practical methods acquire High Dynamic Range (HDR) images by either fusing multiple exposures, or boosting the dynamic range of Standard Dynamic Range (SDR) images. Multiple exposure capture is problematic as it requires longer capture times which can often lead to ghosting problems. The main alternative, inverse tone mapping is an ill-defined problem that is especially challenging as single captured exposures usually contain clipped and quantized values, and are therefore missing substantial amounts of content. To alleviate this, we propose a new approach, High Dynamic Range Thermal (HDRT), for HDR acquisition using a separate, commonly available, thermal infrared (IR) sensor. We propose a novel deep neural method (HDRTNet) which combines IR and SDR content to generate HDR images. HDRTNet learns to exploit IR features linked to the RGB image and the IR-specific parameters are subsequently used in a dual branch method that fuses features at shallow layers. This produces an HDR image that is significantly superior to that generated using naive fusion approaches. To validate our method, we have created the first HDR and thermal dataset, and performed extensive experiments comparing HDRTNet with the state-of-the-art. We show substantial quantitative and qualitative quality improvements on both over- and under-exposed images, showing that our approach is robust to capturing in multiple different lighting conditions.
Read more6/11/2024