Retinexmamba: Retinex-based Mamba for Low-light Image Enhancement

Read original: arXiv:2405.03349 - Published 5/21/2024 by Jiesong Bai, Yuhao Yin, Qiyuan He, Yuanxian Li, Xiaofeng Zhang

🖼️

Overview

The paper introduces a new architecture called "RetinexMamba" for enhancing low-light images
It combines the strengths of traditional Retinex methods and advanced deep learning techniques like Retinexformer
RetinexMamba aims to address the limitations of existing approaches in terms of noise management, detail preservation, and interpretability

Plain English Explanation

Enhancing low-light images is an important challenge in the field of image processing. Traditional Retinex methods try to mimic how the human eye perceives brightness and color, but they can struggle with noise and preserving details in very dark conditions. More advanced deep learning techniques like Retinexformer have shown promise, but they still face issues with interpretability and suboptimal enhancement effects.

The RetinexMamba architecture introduced in this paper combines the best of both worlds. It retains the physical intuitiveness of Retinex methods while leveraging the computational power of deep learning and State Space Models (SSMs) to improve processing speed. RetinexMamba features innovative illumination estimators and damage restorer mechanisms to maintain image quality during enhancement. It also replaces a key component of Retinexformer with a new "Fused-Attention" mechanism, making the model more interpretable.

Experiments on a standard low-light image dataset show that RetinexMamba outperforms other deep learning approaches based on Retinex theory, both in quantitative metrics and visual quality. This suggests the architecture is effective at enhancing low-light images while addressing the limitations of previous methods.

Technical Explanation

The paper builds on the foundations of traditional Retinex methods and the more recent Retinexformer deep learning approach. Retinex methods decompose images into illumination and reflection components, but struggle with noise and detail preservation in low-light conditions. Retinexformer enhances the illumination estimation process using self-attention mechanisms, but faces challenges with interpretability and suboptimal enhancement.

To overcome these limitations, the authors propose the RetinexMamba architecture. RetinexMamba integrates the physical intuitiveness of Retinex methods with the deep learning framework of Retinexformer, leveraging the computational efficiency of State Space Models (SSMs) to improve processing speed. The key innovations include:

Innovative illumination estimators and damage restorer mechanisms to maintain image quality during enhancement
Replacement of the Illumination-Guided Multi-Head Attention (IG-MSA) in Retinexformer with a new "Fused-Attention" mechanism, improving interpretability

Experimental evaluations on the LOL dataset show that RetinexMamba outperforms existing deep learning approaches based on Retinex theory in both quantitative and qualitative metrics, confirming its effectiveness and superiority in enhancing low-light images.

Critical Analysis

The paper provides a compelling solution to the challenge of low-light image enhancement, combining the strengths of traditional and deep learning approaches. However, the authors acknowledge some limitations and areas for further research:

The interpretability of the RetinexMamba model, while improved over Retinexformer, could still be enhanced further to better understand the internal workings of the system.
The experiments were conducted on a single dataset (LOL), and testing the model's generalization to other low-light image datasets would be valuable.
Potential extensions could explore the integration of RetinexMamba with other multimodal fusion techniques or its application to medical image enhancement.

Overall, the RetinexMamba architecture presents a promising advance in the field of low-light image enhancement, leveraging the strengths of both traditional and deep learning approaches. Further research to address the noted limitations and explore additional applications could contribute to the ongoing development of efficient and interpretable vision Mamba models for various domains.

Conclusion

The RetinexMamba architecture introduced in this paper offers a novel solution for enhancing low-light images. By combining the physical intuitiveness of traditional Retinex methods with the computational power of deep learning and State Space Models, RetinexMamba addresses the limitations of existing approaches in terms of noise management, detail preservation, and interpretability. Experimental results demonstrate the effectiveness and superiority of this architecture, paving the way for further advancements in the field of low-light image enhancement and contributing to the broader development of efficient and interpretable vision Mamba models for diverse applications.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

🖼️

Retinexmamba: Retinex-based Mamba for Low-light Image Enhancement

Jiesong Bai, Yuhao Yin, Qiyuan He, Yuanxian Li, Xiaofeng Zhang

In the field of low-light image enhancement, both traditional Retinex methods and advanced deep learning techniques such as Retinexformer have shown distinct advantages and limitations. Traditional Retinex methods, designed to mimic the human eye's perception of brightness and color, decompose images into illumination and reflection components but struggle with noise management and detail preservation under low light conditions. Retinexformer enhances illumination estimation through traditional self-attention mechanisms, but faces challenges with insufficient interpretability and suboptimal enhancement effects. To overcome these limitations, this paper introduces the RetinexMamba architecture. RetinexMamba not only captures the physical intuitiveness of traditional Retinex methods but also integrates the deep learning framework of Retinexformer, leveraging the computational efficiency of State Space Models (SSMs) to enhance processing speed. This architecture features innovative illumination estimators and damage restorer mechanisms that maintain image quality during enhancement. Moreover, RetinexMamba replaces the IG-MSA (Illumination-Guided Multi-Head Attention) in Retinexformer with a Fused-Attention mechanism, improving the model's interpretability. Experimental evaluations on the LOL dataset show that RetinexMamba outperforms existing deep learning approaches based on Retinex theory in both quantitative and qualitative metrics, confirming its effectiveness and superiority in enhancing low-light images.

5/21/2024

LLEMamba: Low-Light Enhancement via Relighting-Guided Mamba with Deep Unfolding Network

Xuanqi Zhang, Haijin Zeng, Jinwang Pan, Qiangqiang Shen, Yongyong Chen

Transformer-based low-light enhancement methods have yielded promising performance by effectively capturing long-range dependencies in a global context. However, their elevated computational demand limits the scalability of multiple iterations in deep unfolding networks, and hence they have difficulty in flexibly balancing interpretability and distortion. To address this issue, we propose a novel Low-Light Enhancement method via relighting-guided Mamba with a deep unfolding network (LLEMamba), whose theoretical interpretability and fidelity are guaranteed by Retinex optimization and Mamba deep priors, respectively. Specifically, our LLEMamba first constructs a Retinex model with deep priors, embedding the iterative optimization process based on the Alternating Direction Method of Multipliers (ADMM) within a deep unfolding network. Unlike Transformer, to assist the deep unfolding framework with multiple iterations, the proposed LLEMamba introduces a novel Mamba architecture with lower computational complexity, which not only achieves light-dependent global visual context for dark images during reflectance relight but also optimizes to obtain more stable closed-form solutions. Experiments on the benchmarks show that LLEMamba achieves superior quantitative evaluations and lower distortion visual results compared to existing state-of-the-art methods.

6/4/2024

MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space

Jiangwei Weng, Zhiqiang Yan, Ying Tai, Jianjun Qian, Jian Yang, Jun Li

Recent advances in low light image enhancement have been dominated by Retinex-based learning framework, leveraging convolutional neural networks (CNNs) and Transformers. However, the vanilla Retinex theory primarily addresses global illumination degradation and neglects local issues such as noise and blur in dark conditions. Moreover, CNNs and Transformers struggle to capture global degradation due to their limited receptive fields. While state space models (SSMs) have shown promise in the long-sequence modeling, they face challenges in combining local invariants and global context in visual data. In this paper, we introduce MambaLLIE, an implicit Retinex-aware low light enhancer featuring a global-then-local state space design. We first propose a Local-Enhanced State Space Module (LESSM) that incorporates an augmented local bias within a 2D selective scan mechanism, enhancing the original SSMs by preserving local 2D dependency. Additionally, an Implicit Retinex-aware Selective Kernel module (IRSK) dynamically selects features using spatially-varying operations, adapting to varying inputs through an adaptive kernel selection process. Our Global-then-Local State Space Block (GLSSB) integrates LESSM and IRSK with LayerNorm as its core. This design enables MambaLLIE to achieve comprehensive global long-range modeling and flexible local feature aggregation. Extensive experiments demonstrate that MambaLLIE significantly outperforms state-of-the-art CNN and Transformer-based methods. Project Page: https://mamballie.github.io/anon/

5/28/2024

Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement

Xianmin Chen, Peiliang Huang, Xiaoxu Feng, Dingwen Zhang, Longfei Han, Junwei Han

Low-light image enhancement, particularly in cross-domain tasks such as mapping from the raw domain to the sRGB domain, remains a significant challenge. Many deep learning-based methods have been developed to address this issue and have shown promising results in recent years. However, single-stage methods, which attempt to unify the complex mapping across both domains, leading to limited denoising performance. In contrast, two-stage approaches typically decompose a raw image with color filter arrays (CFA) into a four-channel RGGB format before feeding it into a neural network. However, this strategy overlooks the critical role of demosaicing within the Image Signal Processing (ISP) pipeline, leading to color distortions under varying lighting conditions, especially in low-light scenarios. To address these issues, we design a novel Mamba scanning mechanism, called RAWMamba, to effectively handle raw images with different CFAs. Furthermore, we present a Retinex Decomposition Module (RDM) grounded in Retinex prior, which decouples illumination from reflectance to facilitate more effective denoising and automatic non-linear exposure correction. By bridging demosaicing and denoising, better raw image enhancement is achieved. Experimental evaluations conducted on public datasets SID and MCR demonstrate that our proposed RAWMamba achieves state-of-the-art performance on cross-domain mapping.

9/12/2024