Ptychographic non-line-of-sight imaging for depth-resolved visualization of hidden objects
2405.11115
0
0
📉
Abstract
Non-line-of-sight (NLOS) imaging enables the visualization of objects hidden from direct view, with applications in surveillance, remote sensing, and light detection and ranging. Here, we introduce a NLOS imaging technique termed ptychographic NLOS (pNLOS), which leverages coded ptychography for depth-resolved imaging of obscured objects. Our approach involves scanning a laser spot on a wall to illuminate the hidden objects in an obscured region. The reflected wavefields from these objects then travel back to the wall, get modulated by the wall's complex-valued profile, and the resulting diffraction patterns are captured by a camera. By modulating the object wavefields, the wall surface serves the role of the coded layer as in coded ptychography. As we scan the laser spot to different positions, the reflected object wavefields on the wall translate accordingly, with the shifts varying for objects at different depths. This translational diversity enables the acquisition of a set of modulated diffraction patterns referred to as a ptychogram. By processing the ptychogram, we recover both the objects at different depths and the modulation profile of the wall surface. Experimental results demonstrate high-resolution, high-fidelity imaging of hidden objects, showcasing the potential of pNLOS for depth-aware vision beyond the direct line of sight.
Create account to get full access
Overview
- Non-line-of-sight (NLOS) imaging enables visualizing objects hidden from direct view, with applications in surveillance, remote sensing, and light detection and ranging.
- The paper introduces a NLOS imaging technique called ptychographic NLOS (pNLOS), which uses coded ptychography for depth-resolved imaging of obscured objects.
- pNLOS involves scanning a laser spot on a wall to illuminate hidden objects, capturing the reflected wavefields modulated by the wall's complex-valued profile, and processing the resulting diffraction patterns to recover the hidden objects and wall modulation profile.
Plain English Explanation
pNLOS is a technique that allows you to see things that are hidden from your direct line of sight. Imagine you're in a room, and there's a wall blocking your view of something on the other side. With pNLOS, you can use a laser to shine light on the wall, and the light that bounces off the hidden objects on the other side will get captured by a camera. By analyzing how the light waves are changed by the wall, the researchers can reconstruct what the hidden objects look like and how far away they are.
This is useful for all kinds of applications, like surveillance, remote sensing, and 3D mapping. For example, it could help you see around corners or through dense foliage, which could be important for self-driving cars or search-and-rescue missions. It's like having the ability to see "behind the curtain" without actually being able to look directly at what's there.
Technical Explanation
The pNLOS technique leverages coded ptychography, which involves using a coded layer (in this case, the wall surface) to modulate the wavefields from hidden objects. The researchers scan a laser spot across the wall, and the reflected wavefields from the hidden objects get modulated by the wall's complex-valued profile.
As the laser spot moves, the reflected object wavefields on the wall translate, with the shifts varying for objects at different depths. This translational diversity enables the acquisition of a set of modulated diffraction patterns, referred to as a ptychogram. By processing the ptychogram, the researchers can recover both the hidden objects at different depths and the modulation profile of the wall surface.
The experimental results demonstrate high-resolution, high-fidelity imaging of hidden objects, showcasing the potential of pNLOS for depth-aware vision beyond the direct line of sight.
Critical Analysis
The paper presents a promising NLOS imaging technique, but there are a few potential limitations and areas for further research. First, the method requires a wall with a known, complex-valued profile, which may not always be available in real-world scenarios. Developing techniques to handle unknown or partially known wall profiles could expand the applicability of pNLOS.
Additionally, the current setup may be sensitive to environmental factors, such as vibrations or changes in the wall surface over time. Investigating the robustness of the technique to these factors would be important for practical deployments.
Finally, the paper focuses on proof-of-concept experiments in a controlled laboratory setting. Exploring the performance of pNLOS in more realistic, complex environments with varying levels of occlusion and clutter would help assess its practical feasibility and limitations.
Conclusion
The pNLOS technique introduced in this paper represents a significant advancement in NLOS imaging, enabling high-resolution, depth-aware visualization of hidden objects. By leveraging coded ptychography, the method can recover both the object information and the wall modulation profile, overcoming limitations of previous NLOS approaches.
If the identified challenges can be addressed, pNLOS could have a transformative impact on applications ranging from surveillance and remote sensing to robotic navigation and search-and-rescue operations, where the ability to see beyond the direct line of sight can be invaluable.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot
Shenbagaraj Kannapiran, Sreenithy Chandran, Suren Jayasuriya, Spring Berman
0
0
The study of non-line-of-sight (NLOS) imaging is growing due to its many potential applications, including rescue operations and pedestrian detection by self-driving cars. However, implementing NLOS imaging on a moving camera remains an open area of research. Existing NLOS imaging methods rely on time-resolved detectors and laser configurations that require precise optical alignment, making it difficult to deploy them in dynamic environments. This work proposes a data-driven approach to NLOS imaging, PathFinder, that can be used with a standard RGB camera mounted on a small, power-constrained mobile robot, such as an aerial drone. Our experimental pipeline is designed to accurately estimate the 2D trajectory of a person who moves in a Manhattan-world environment while remaining hidden from the camera's field-of-view. We introduce a novel approach to process a sequence of dynamic successive frames in a line-of-sight (LOS) video using an attention-based neural network that performs inference in real-time. The method also includes a preprocessing selection metric that analyzes images from a moving camera which contain multiple vertical planar surfaces, such as walls and building facades, and extracts planes that return maximum NLOS information. We validate the approach on in-the-wild scenes using a drone for video capture, thus demonstrating low-cost NLOS imaging in dynamic capture environments.
4/9/2024
↗️
A high-performance reconstruction method for partially coherent ptychography
Wenhui Xu, Shoucong Ning, Pengju Sheng, Huixiang Lin, Angus I Kirkland, Yong Peng, Fucai Zhang
0
0
Ptychography is now integrated as a tool in mainstream microscopy allowing quantitative and high-resolution imaging capabilities over a wide field of view. However, its ultimate performance is inevitably limited by the available coherent flux when implemented using electrons or laboratory X-ray sources. We present a universal reconstruction algorithm with high tolerance to low coherence for both far-field and near-field ptychography. The approach is practical for partial temporal and spatial coherence and requires no prior knowledge of the source properties. Our initial visible-light and electron data show that the method can dramatically improve the reconstruction quality and accelerate the convergence rate of the reconstruction. The approach also integrates well into existing ptychographic engines. It can also improve mixed-state and numerical monochromatisation methods, requiring a smaller number of coherent modes or lower dimensionality of Krylov subspace while providing more stable and faster convergence. We propose that this approach could have significant impact on ptychography of weakly scattering samples.
6/12/2024
PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar
Tzofi Klinghoffer, Xiaoyu Xiang, Siddharth Somasundaram, Yuchen Fan, Christian Richardt, Ramesh Raskar, Rakesh Ranjan
0
0
3D reconstruction from a single-view is challenging because of the ambiguity from monocular cues and lack of information about occluded regions. Neural radiance fields (NeRF), while popular for view synthesis and 3D reconstruction, are typically reliant on multi-view images. Existing methods for single-view 3D reconstruction with NeRF rely on either data priors to hallucinate views of occluded regions, which may not be physically accurate, or shadows observed by RGB cameras, which are difficult to detect in ambient light and low albedo backgrounds. We propose using time-of-flight data captured by a single-photon avalanche diode to overcome these limitations. Our method models two-bounce optical paths with NeRF, using lidar transient data for supervision. By leveraging the advantages of both NeRF and two-bounce light measured by lidar, we demonstrate that we can reconstruct visible and occluded geometry without data priors or reliance on controlled ambient lighting or scene albedo. In addition, we demonstrate improved generalization under practical constraints on sensor spatial- and temporal-resolution. We believe our method is a promising direction as single-photon lidars become ubiquitous on consumer devices, such as phones, tablets, and headsets.
4/8/2024
🔮
First path component power based NLOS mitigation in UWB positioning system
Marcin Kolakowski, Jozef Modelski
0
0
The paper describes an NLOS (Non-Line-of-Sight) mitigation method intended for use in a UWB positioning system. In the proposed method propagation conditions between the localized objects and the anchors forming system infrastructure are classified into one of three categories: LOS (Line-of-Sight), NLOS and severe NLOS. Non-Line-of-Sight detection is conducted based on first path signal component power measurements. For each of the categories, average NLOS inducted time of arrival bias and bias standard deviation have been estimated based on results gathered during a measurement campaign conducted in a fully furnished apartment. To locate a tag, an EKF (Extended Kalman Filter) based algorithm is used. The proposed method of NLOS mitigation consists in correcting measurement results obtained in NLOS conditions and lowering their significance in a tag position estimation process. The paper includes the description of the method and the results of the conducted experiments.
4/1/2024