Panoramic single-pixel imaging with megapixel resolution based on rotational subdivision
0
🤿
Sign in to get full access
Overview
- Single-pixel imaging (SPI) uses a single-pixel detector to capture images, unlike conventional cameras with many pixels.
- Catadioptric panoramic ghost imaging (CPGI) extends the application of SPI to high-performance panoramic imaging with a wide field of view (FOV).
- However, the resolution of CPGI is limited by the hardware parameters of the digital micromirror device (DMD) used.
- To overcome this resolution limitation, the researchers propose a new technique called panoramic SPI based on rotational subdivision (RSPSI).
Plain English Explanation
The paper presents a new imaging technique called panoramic SPI based on rotational subdivision (RSPSI) that can capture ultra-high-resolution panoramic images using a single-pixel detector.
Conventional cameras have many individual pixels that work together to capture an image. In contrast, single-pixel imaging (SPI) uses just a single detector to capture an entire image. This SPI approach has some advantages, like the ability to work with limited hardware.
The researchers previously developed a technique called catadioptric panoramic ghost imaging (CPGI) that applies SPI to panoramic imaging, allowing for wide-angle, high-performance imaging. However, the resolution of CPGI is limited by the hardware used.
To overcome this resolution limitation, the researchers created a new technique called RSPSI. The key idea is to capture the entire panoramic scene by rotating a mirror, so that a single pattern covering a small sub-region can be used to modulate the entire panoramic view. Then, by combining multiple sub-region images captured over time, they can reconstruct a very high-resolution panoramic image.
Experiments show that RSPSI can achieve a panoramic image resolution of over 5.6 million pixels, more than 9 times higher than the CPGI approach using the same hardware. This could enable applications requiring ultra-high-resolution panoramic imaging.
Technical Explanation
The RSPSI technique works by rotating a 45-degree tilted mirror to capture the entire panoramic scene. A single pattern projected onto the mirror only covers a small sub-field-of-view (sub-FOV), but as the mirror rotates, this pattern can modulate the entire panoramic FOV.
By capturing a sequence of these sub-FOV images over time and then reconstructing them with precise horizontal shifting, the researchers can assemble a final panoramic image with very high resolution - over 10,000 pixels wide in their experiments. This is much higher than the resolution limit of the CPGI approach, which is constrained by the hardware parameters of the digital micromirror device (DMD).
The key technical innovation is this "rotational subdivision" approach that allows a small sub-FOV pattern to be used to capture the entire panoramic scene. Combined with temporal resolution subdivision, this enables reconstructing a megapixel-scale panoramic image from the single-pixel detector data.
Critical Analysis
The RSPSI technique represents an impressive advance in single-pixel imaging, demonstrating the ability to capture ultra-high-resolution panoramic images well beyond previous SPI approaches. However, the paper does not extensively discuss potential limitations or areas for further improvement.
One potential limitation is the reliance on a rotating mirror, which could introduce mechanical complexity and potential reliability issues compared to a static system. The paper also does not explore how the resolution and image quality might scale with different hardware parameters, such as the DMD resolution or the optical system design.
Additionally, the paper does not provide a detailed analysis of the computational and time requirements for the image reconstruction process. For practical applications, these aspects may be important considerations.
Overall, the RSPSI technique is a significant technological advancement, but further research is needed to fully understand its capabilities, limitations, and potential real-world applications.
Conclusion
The panoramic SPI based on rotational subdivision (RSPSI) technique presented in this paper represents a major step forward in single-pixel imaging, demonstrating the ability to capture ultra-high-resolution panoramic images using a single-pixel detector.
By leveraging a rotating mirror and temporal subdivision, RSPSI can overcome the resolution limitations of previous SPI panoramic imaging approaches. The researchers were able to achieve a panoramic image resolution of over 5.6 million pixels, more than 9 times higher than what was possible with the earlier CPGI method.
This advance could enable new applications that require detailed, wide-angle imaging, such as surveillance, navigation, or virtual reality. Further research is needed to fully understand the capabilities, limitations, and practical considerations of the RSPSI approach, but it represents an exciting development in the field of computational imaging.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🤿
0
Panoramic single-pixel imaging with megapixel resolution based on rotational subdivision
Huan Cui, Jie Cao, Haoyu Zhang, Chang Zhou, Haifeng Yao, Yingbo Wang, Qun Hao
Single-pixel imaging (SPI) using a single-pixel detector is an unconventional imaging method, which has great application prospects in many fields to realize high-performance imaging. In especial, the recent proposed catadioptric panoramic ghost imaging (CPGI) extends the application potential of SPI to high-performance imaging at a wide field of view (FOV) with recent growing demands. However, the resolution of CPGI is limited by hardware parameters of the digital micromirror device (DMD), which may not meet ultrahigh-resolution panoramic imaging needs that require detailed information. Therefore, to overcome the resolution limitation of CPGI, we propose a panoramic SPI based on rotational subdivision (RSPSI). The key of the proposed RSPSI is to obtain the entire panoramic scene by the rotation-scanning with a rotating mirror tilted 45{deg}, so that one single pattern that only covers one sub-Fov with a small FOV can complete a uninterrupted modulation on the entire panoramic FOV during a once-through pattern projection. Then, based on temporal resolution subdivision, images sequence of sub-Fovs subdivided from the entire panoramic FOV can be reconstructed with pixels-level or even subpixels-level horizontal shifting adjacently. Experimental results using a proof-of-concept setup show that the panoramic image can be obtained with 10428*543 of 5,662,404 pixels, which is more than 9.6 times higher than the resolution limit of the CPGI using the same DMD. To our best knowledge, the RSPSI is the first to achieve a megapixel resolution via SPI, which can provide potential applications in fields requiring the imaging with ultrahigh-resolution and wide FOV.
Read more7/30/2024
0
Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers
Qi Jiang, Shaohua Gao, Yao Gao, Kailun Yang, Zhonghua Yi, Hao Shi, Lei Sun, Kaiwei Wang
High-quality panoramic images with a Field of View (FoV) of 360{deg} are essential for contemporary panoramic computer vision tasks. However, conventional imaging systems come with sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile and wearable applications where thin and portable, minimalist imaging systems are desired. In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to achieve minimalist and high-quality panoramic imaging. With less than three spherical lenses, a Minimalist Panoramic Imaging Prototype (MPIP) is constructed based on the design of the Panoramic Annular Lens (PAL), but with low-quality imaging results due to aberrations and small image plane size. We propose two pipelines, i.e. Aberration Correction (AC) and Super-Resolution and Aberration Correction (SR&AC), to solve the image quality problems of MPIP, with imaging sensors of small and large pixel size, respectively. To leverage the prior information of the optical system, we propose a Point Spread Function (PSF) representation method to produce a PSF map as an additional modality. A PSF-aware Aberration-image Recovery Transformer (PART) is designed as a universal network for the two pipelines, in which the self-attention calculation and feature extraction are guided by the PSF map. We train PART on synthetic image pairs from simulation and put forward the PALHQ dataset to fill the gap of real-world high-quality PAL images for low-level vision. A comprehensive variety of experiments on synthetic and real-world benchmarks demonstrates the impressive imaging results of PCIE and the effectiveness of the PSF representation. We further deliver heuristic experimental findings for minimalist and high-quality panoramic imaging. Our dataset and code will be available at https://github.com/zju-jiangqi/PCIE-PART.
Read more7/8/2024
🧠
0
Pixel super-resolved lensless on-chip sensor with scattering multiplexing
Xuyang Chang, Shaowei Jiang, Yongcun Hu, Liheng Bian
Lensless on-chip microscopy has shown great potential for biomedical imaging due to its large-area and high-throughput imaging capabilities. By combining the pixel super-resolution (PSR) technique, it can improve the resolution beyond the limit of the imaging detector. However, existing PSR techniques are restricted to the feature size and crosstalk of modulation components (such as spatial light modulator), which cannot efficiently encode target information. Besides, the reconstruction algorithms suffer from the trade-off between image quality, reconstruction resolution and computational efficiency. In this work, we constructed a novel integrated lensless on-chip sensor via scattering multiplexing, and reported a robust PSR algorithm for sample reconstruction. The sensor employed a scattering layer as a modulator, which was permanently integrated with the detector. Benefiting from the high-degree-of-freedom reconstruction of the scattering layer, we realized fine wavefront modulation with a small feature size. The integration engineering avoided repetitious calibration and reduce the measurement complexity. The reported PSR algorithm combines both model-driven and data-driven strategies to efficiently exploit the high-frequency information from the fine modulation. A series of experiments validated that the reported sensor provides a low-cost solution for large-scale microscopic imaging, with significant advantages in resolution, image contrast and noise robustness.
Read more9/6/2024
↗️
0
Dual-Scale Transformer for Large-Scale Single-Pixel Imaging
Gang Qu, Ping Wang, Xin Yuan
Single-pixel imaging (SPI) is a potential computational imaging technique which produces image by solving an illposed reconstruction problem from few measurements captured by a single-pixel detector. Deep learning has achieved impressive success on SPI reconstruction. However, previous poor reconstruction performance and impractical imaging model limit its real-world applications. In this paper, we propose a deep unfolding network with hybrid-attention Transformer on Kronecker SPI model, dubbed HATNet, to improve the imaging quality of real SPI cameras. Specifically, we unfold the computation graph of the iterative shrinkagethresholding algorithm (ISTA) into two alternative modules: efficient tensor gradient descent and hybrid-attention multiscale denoising. By virtue of Kronecker SPI, the gradient descent module can avoid high computational overheads rooted in previous gradient descent modules based on vectorized SPI. The denoising module is an encoder-decoder architecture powered by dual-scale spatial attention for high- and low-frequency aggregation and channel attention for global information recalibration. Moreover, we build a SPI prototype to verify the effectiveness of the proposed method. Extensive experiments on synthetic and real data demonstrate that our method achieves the state-of-the-art performance. The source code and pre-trained models are available at https://github.com/Gang-Qu/HATNet-SPI.
Read more4/9/2024