MambaUIE&SR: Unraveling the Ocean's Secrets with Only 2.8 FLOPs

Overview

• Underwater Image Enhancement (UIE) techniques aim to address the problem of underwater image degradation due to light absorption and scattering.
• Recent approaches have explored both Convolutional Neural Network (CNN)-based and Transformer-based methods, as well as combining CNN and Transformer to effectively combine global and local information for enhancement.
• However, these approaches are still affected by the secondary complexity of the Transformer and cannot maximize performance.
• The state-space model (SSM)-based architecture Mamba has been proposed, which excels in modeling long distances while maintaining linear complexity.

Plain English Explanation

Underwater images often suffer from poor quality due to the way light behaves in water. Light gets absorbed and scattered, making the images look dull and distorted. Researchers have been exploring different ways to enhance these underwater images and improve their quality.

Recently, they've been trying two main approaches: Convolutional Neural Networks (CNNs) and Transformer-based models. CNNs are good at capturing local details, while Transformers can capture global context. Combining these two approaches can help enhance both the local and global information in the images.

However, the Transformer part of these models adds extra complexity, which can limit their overall performance. To address this, researchers have now explored using a state-space model (SSM) architecture called Mamba. This SSM-based model is efficient at handling long-range dependencies while keeping the complexity low.

Technical Explanation

The researchers explore the potential of the SSM-based Mamba model for UIE, from both efficiency and effectiveness perspectives. However, they find that directly applying Mamba leads to poor performance, as it cannot fully utilize the local fine-grained features that are crucial for image enhancement.

To address this, the researchers propose a customized MambaUIE architecture for efficient UIE. They introduce Visual State Space (VSS) blocks to capture global contextual information at the macro level while mining local information at the micro level. They also propose a Dynamic Interaction Block (DIB) and a Spatial feed-forward Network (SGFN) for intra-block feature aggregation of these two types of information.

This allows MambaUIE to efficiently synthesize global and local information while maintaining a very small number of parameters and high accuracy. Experiments on the UIEB dataset show that MambaUIE reduces GFLOPs (a measure of computational complexity) by 67.4% (2.715G) compared to the state-of-the-art method.

Critical Analysis

The paper introduces a novel approach to underwater image enhancement by leveraging the efficient Mamba architecture. However, the researchers acknowledge that directly applying Mamba leads to poor performance because it cannot fully utilize the local fine-grained features, which are crucial for image enhancement.

To address this, the researchers propose customizations to the Mamba architecture, such as the Visual State Space (VSS) blocks, Dynamic Interaction Block (DIB), and Spatial feed-forward Network (SGFN). These customizations allow MambaUIE to effectively capture both global and local information, leading to significant improvements in computational efficiency and accuracy.

While the results are promising, the paper does not explore the potential limitations or drawbacks of the MambaUIE approach. For example, it would be interesting to understand how the model performs on a wider range of underwater image datasets, or how it compares to other state-of-the-art methods in terms of qualitative image enhancement.

Conclusion

This paper presents a novel approach to underwater image enhancement using the efficient Mamba architecture. By customizing the Mamba model with specialized blocks, the researchers were able to capture both global and local information, leading to significant improvements in computational efficiency and accuracy compared to the state-of-the-art.

This work demonstrates the potential of state-space models, like Mamba, for tackling complex computer vision problems, such as underwater image enhancement. The MambaUIE model's ability to efficiently synthesize global and local information while maintaining a small number of parameters could have broader implications for developing high-performance, resource-efficient computer vision systems.