Pixel super-resolved lensless on-chip sensor with scattering multiplexing

Read original: arXiv:2105.14746 - Published 9/6/2024 by Xuyang Chang, Shaowei Jiang, Yongcun Hu, Liheng Bian
Total Score

0

🧠

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • Lensless on-chip microscopy is a promising approach for biomedical imaging, offering large-area and high-throughput capabilities.
  • Pixel super-resolution (PSR) can enhance the resolution beyond the limits of the imaging detector.
  • Existing PSR techniques are restricted by the features and crosstalk of modulation components, and reconstruction algorithms face trade-offs.
  • This work introduces a novel integrated lensless on-chip sensor using scattering multiplexing and a robust PSR algorithm for sample reconstruction.

Plain English Explanation

Lensless on-chip microscopes are an exciting development in biomedical imaging because they can cover large areas and image quickly. By combining this technology with a technique called pixel super-resolution, the resolution of the images can be improved beyond what the imaging detector is normally capable of.

However, existing pixel super-resolution methods have limitations. The components used to modify the light (like a spatial light modulator) have constraints on their feature size and how much they can interfere with each other. This means they can't efficiently capture all the details in the target being imaged. The algorithms used to reconstruct the high-resolution images also have to balance image quality, resolution, and computational efficiency, which can be challenging.

In this research, the team developed a new type of lensless on-chip sensor that uses a scattering layer as the modulator instead of traditional components. This allows for finer control over the light wavefront, capturing more high-frequency information. They also came up with a new pixel super-resolution algorithm that combines model-based and data-driven approaches to efficiently reconstruct the high-resolution images.

Through experiments, the researchers showed that this new sensor and algorithm provide a low-cost solution for large-scale microscopic imaging, with significant improvements in resolution, image contrast, and noise robustness compared to existing methods.

Technical Explanation

The researchers constructed a novel integrated lensless on-chip sensor using scattering multiplexing. Unlike previous approaches that used modulation components like spatial light modulators, this sensor employed a scattering layer that was permanently integrated with the detector.

The scattering layer provided a high-degree-of-freedom for wavefront modulation, enabling fine control with a small feature size. This avoided the limitations of traditional modulation components related to feature size and crosstalk. The integrated design also eliminated the need for repetitive calibration, reducing measurement complexity.

To reconstruct high-resolution images from the sensor's output, the researchers developed a robust pixel super-resolution (PSR) algorithm. This algorithm combined model-driven and data-driven strategies to efficiently exploit the high-frequency information encoded by the fine wavefront modulation of the scattering layer.

Through a series of experiments, the team validated that this integrated lensless on-chip sensor and PSR algorithm provide a low-cost solution for large-scale microscopic imaging. The system demonstrated significant advantages over existing approaches in terms of resolution, image contrast, and noise robustness.

Critical Analysis

The researchers acknowledge that their scattering-based modulation approach has some limitations. The scattering pattern is fixed after the fabrication process, so the modulation cannot be dynamically adjusted like with a spatial light modulator. This may restrict the system's flexibility and adaptability to different imaging scenarios.

Additionally, the paper does not provide a detailed analysis of the trade-offs between the reconstruction resolution, computational efficiency, and image quality of the PSR algorithm. While the results show improvements, further investigation into these performance factors would help users better understand the capabilities and limitations of the approach.

It would also be valuable for the authors to discuss potential challenges in scaling up this lensless on-chip sensor technology for real-world biomedical applications. Factors like field-of-view, imaging depth, and integration with existing workflows would be important considerations.

Overall, this research represents an innovative step forward in lensless on-chip microscopy, leveraging scattering and advanced reconstruction algorithms to enhance resolution and performance. Further work to address the identified limitations and explore practical implementation details would help solidify the benefits of this approach for biomedical imaging.

Conclusion

This study presents a novel integrated lensless on-chip sensor that uses scattering multiplexing to enable high-resolution imaging beyond the limits of the detector. By employing a permanently integrated scattering layer as the modulator, the sensor avoids the constraints of traditional components and allows for fine wavefront control.

The researchers also developed a robust pixel super-resolution algorithm that efficiently exploits the high-frequency information encoded by the scattering layer. Experimental results demonstrate that this integrated sensor and algorithm provide a low-cost solution for large-scale microscopic imaging, with significant improvements in resolution, contrast, and noise robustness compared to existing lensless on-chip microscopy approaches.

This work represents an important advancement in the field of lensless on-chip microscopy, opening up new possibilities for high-performance, large-area biomedical imaging at a lower cost. Further development and real-world validation of this technology could have far-reaching impacts in areas like disease diagnosis, drug discovery, and biological research.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

🧠

Total Score

0

Pixel super-resolved lensless on-chip sensor with scattering multiplexing

Xuyang Chang, Shaowei Jiang, Yongcun Hu, Liheng Bian

Lensless on-chip microscopy has shown great potential for biomedical imaging due to its large-area and high-throughput imaging capabilities. By combining the pixel super-resolution (PSR) technique, it can improve the resolution beyond the limit of the imaging detector. However, existing PSR techniques are restricted to the feature size and crosstalk of modulation components (such as spatial light modulator), which cannot efficiently encode target information. Besides, the reconstruction algorithms suffer from the trade-off between image quality, reconstruction resolution and computational efficiency. In this work, we constructed a novel integrated lensless on-chip sensor via scattering multiplexing, and reported a robust PSR algorithm for sample reconstruction. The sensor employed a scattering layer as a modulator, which was permanently integrated with the detector. Benefiting from the high-degree-of-freedom reconstruction of the scattering layer, we realized fine wavefront modulation with a small feature size. The integration engineering avoided repetitious calibration and reduce the measurement complexity. The reported PSR algorithm combines both model-driven and data-driven strategies to efficiently exploit the high-frequency information from the fine modulation. A series of experiments validated that the reported sensor provides a low-cost solution for large-scale microscopic imaging, with significant advantages in resolution, image contrast and noise robustness.

Read more

9/6/2024

👨‍🏫

Total Score

0

Scattering-induced entropy boost for highly-compressed optical sensing and encryption

Xinrui Zhan, Xuyang Chang, Daoyu Li, Rong Yan, Yinuo Zhang, Liheng Bian

Image sensing often relies on a high-quality machine vision system with a large field of view and high resolution. It requires fine imaging optics, has high computational costs, and requires a large communication bandwidth between image sensors and computing units. In this paper, we propose a novel image-free sensing framework for resource-efficient image classification, where the required number of measurements can be reduced by up to two orders of magnitude. In the proposed framework for single-pixel detection, the optical field for a target is first scattered by an optical diffuser and then two-dimensionally modulated by a spatial light modulator. The optical diffuser simultaneously serves as a compressor and an encryptor for the target information, effectively narrowing the field of view and improving the system's security. The one-dimensional sequence of intensity values, which is measured with time-varying patterns on the spatial light modulator, is then used to extract semantic information based on end-to-end deep learning. The proposed sensing framework is shown to obtain over a 95% accuracy at sampling rates of 1% and 5% for classification on the MNIST dataset and the recognition of Chinese license plates, respectively, and the framework is up to 24% more efficient than the approach without an optical diffuser. The proposed framework represents a significant breakthrough in high-throughput machine intelligence for scene analysis with low bandwidth, low costs, and strong encryption.

Read more

9/9/2024

Towards Physics-informed Cyclic Adversarial Multi-PSF Lensless Imaging
Total Score

0

Towards Physics-informed Cyclic Adversarial Multi-PSF Lensless Imaging

Abeer Banerjee, Sanjay Singh

Lensless imaging has emerged as a promising field within inverse imaging, offering compact, cost-effective solutions with the potential to revolutionize the computational camera market. By circumventing traditional optical components like lenses and mirrors, novel approaches like mask-based lensless imaging eliminate the need for conventional hardware. However, advancements in lensless image reconstruction, particularly those leveraging Generative Adversarial Networks (GANs), are hindered by the reliance on data-driven training processes, resulting in network specificity to the Point Spread Function (PSF) of the imaging system. This necessitates a complete retraining for minor PSF changes, limiting adaptability and generalizability across diverse imaging scenarios. In this paper, we introduce a novel approach to multi-PSF lensless imaging, employing a dual discriminator cyclic adversarial framework. We propose a unique generator architecture with a sparse convolutional PSF-aware auxiliary branch, coupled with a forward model integrated into the training loop to facilitate physics-informed learning to handle the substantial domain gap between lensless and lensed images. Comprehensive performance evaluation and ablation studies underscore the effectiveness of our model, offering robust and adaptable lensless image reconstruction capabilities. Our method achieves comparable performance to existing PSF-agnostic generative methods for single PSF cases and demonstrates resilience to PSF changes without the need for retraining.

Read more

7/10/2024

Neuromorphic Imaging with Super-Resolution
Total Score

0

Neuromorphic Imaging with Super-Resolution

Pei Zhang, Shuo Zhu, Chutian Wang, Yaping Zhao, Edmund Y. Lam

Neuromorphic imaging is a bio-inspired technique that imitates the human retina to sense variations in a dynamic scene. It responds to pixel-level brightness changes by asynchronous streaming events and boasts microsecond temporal precision over a high dynamic range, yielding blur-free recordings under extreme illumination. Nevertheless, such a modality falls short in spatial resolution and leads to a low level of visual richness and clarity. Pursuing hardware upgrades is expensive and might cause compromised performance due to more burdens on computational requirements. Another option is to harness offline, plug-in-play neuromorphic super-resolution solutions. However, existing ones, which demand substantial sample volumes for lengthy training on massive computing resources, are largely restricted by real data availability owing to the current imperfect high-resolution devices, as well as the randomness and variability of motion. To tackle these challenges, we introduce the first self-supervised neuromorphic super-resolution prototype. It can be self-adaptive to per input source from any low-resolution camera to estimate an optimal, high-resolution counterpart of any scale, without the need of side knowledge and prior training. Evaluated on downstream event-driven tasks, such a simple yet effective method can obtain competitive results against the state-of-the-arts, significantly promoting flexibility but not sacrificing accuracy. It also delivers enhancements for inferior natural images and optical micrographs acquired under non-ideal imaging conditions, breaking through the limitations that are challenging to overcome with traditional frame techniques. In the current landscape where the use of high-resolution cameras for event-based sensing remains an open debate, our solution serves as a cost-efficient and practical alternative, paving the way for more intelligent imaging systems.

Read more

7/9/2024