A Minimal Set of Parameters Based Depth-Dependent Distortion Model and Its Calibration Method for Stereo Vision Systems
2404.19242
0
0
📈
Abstract
Depth position highly affects lens distortion, especially in close-range photography, which limits the measurement accuracy of existing stereo vision systems. Moreover, traditional depth-dependent distortion models and their calibration methods have remained complicated. In this work, we propose a minimal set of parameters based depth-dependent distortion model (MDM), which considers the radial and decentering distortions of the lens to improve the accuracy of stereo vision systems and simplify their calibration process. In addition, we present an easy and flexible calibration method for the MDM of stereo vision systems with a commonly used planar pattern, which requires cameras to observe the planar pattern in different orientations. The proposed technique is easy to use and flexible compared with classical calibration techniques for depth-dependent distortion models in which the lens must be perpendicular to the planar pattern. The experimental validation of the MDM and its calibration method showed that the MDM improved the calibration accuracy by 56.55% and 74.15% compared with the Li's distortion model and traditional Brown's distortion model. Besides, an iteration-based reconstruction method is proposed to iteratively estimate the depth information in the MDM during three-dimensional reconstruction. The results showed that the accuracy of the iteration-based reconstruction method was improved by 9.08% compared with that of the non-iteration reconstruction method.
Create account to get full access
Overview
- The paper proposes a new depth-dependent distortion model (MDM) to improve the accuracy of stereo vision systems, which are used for 3D reconstruction.
- Existing stereo vision systems suffer from lens distortion, especially at close range, which limits their measurement accuracy.
- The MDM considers both radial and decentering distortions to better model lens distortion.
- The paper also presents a simplified calibration method for the MDM using a planar pattern, which is easier to use than traditional calibration techniques.
- An iteration-based 3D reconstruction method is proposed to further improve the accuracy of the system.
Plain English Explanation
Cameras and lenses can distort the images they capture, especially when objects are very close to the camera. This can be a problem for stereo vision systems, which use two or more cameras to estimate the 3D shape of objects.
The researchers in this paper developed a new way to model and correct this lens distortion, called the MDM. The MDM considers both radial distortion (where the edges of the image appear curved) and decentering distortion (where the image is shifted off-center). By accounting for these types of distortion, the MDM can improve the accuracy of 3D measurements made by stereo vision systems.
The researchers also created a simpler way to calibrate the MDM using a flat, checkered pattern that the cameras can observe from different angles. This is easier than the complex calibration procedures typically required for depth-dependent distortion models.
Finally, the paper presents a method to iteratively refine the 3D reconstruction during the estimation process, further improving the accuracy compared to non-iterative approaches. This helps overcome the challenges of lens distortion and produce more precise 3D models.
Technical Explanation
The paper proposes a minimal set of parameters based depth-dependent distortion model (MDM) that considers both radial and decentering lens distortions. This improves the accuracy of stereo vision systems compared to existing depth-dependent distortion models like Li's model and the traditional Brown's model.
The researchers present a flexible calibration method for the MDM that uses a planar calibration pattern. This is simpler than classical calibration techniques that require the lens to be perpendicular to the pattern. The cameras only need to observe the pattern from different orientations.
Experimental validation showed the MDM improved calibration accuracy by over 50% compared to the Li and Brown models. The researchers also proposed an iteration-based 3D reconstruction method that leverages the MDM to further enhance reconstruction accuracy by 9% compared to non-iterative approaches.
Critical Analysis
The paper provides a solid technical contribution by introducing the MDM distortion model and a practical calibration method. The experiments demonstrate clear improvements in measurement accuracy for stereo vision systems.
However, the paper does not address potential limitations or caveats of the proposed approach. For example, it is unclear how the MDM and calibration method would perform under more challenging conditions, such as large depth ranges, complex scenes, or lower-quality cameras.
Additionally, the paper does not compare the MDM to more recent distortion models or calibration techniques that may have been developed since its publication. An analysis of how the MDM stands up against the state-of-the-art would strengthen the claims of novelty and impact.
Further research could also investigate the computational complexity and real-time performance of the MDM and iteration-based reconstruction, as these factors are crucial for practical applications of stereo vision.
Overall, the paper presents a promising technical advance, but deeper analysis of the approach's limitations and comparison to recent works would strengthen the contribution.
Conclusion
This paper introduces a new depth-dependent distortion model (MDM) and a simplified calibration method to improve the accuracy of stereo vision systems. By accounting for both radial and decentering lens distortions, the MDM outperforms existing distortion models in terms of calibration and 3D reconstruction accuracy.
The proposed calibration technique, which uses a planar pattern observed from different angles, makes the process easier and more flexible compared to traditional methods. The iteration-based 3D reconstruction further enhances the accuracy of the system.
These advancements have the potential to significantly improve the performance of stereo vision in applications like robotics, augmented reality, and 3D mapping, where precise 3D reconstruction is crucial. Future work should explore the MDM's robustness to challenging conditions and compare it to state-of-the-art distortion models and calibration techniques.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🌀
Single-image camera calibration with model-free distortion correction
Katia Genovese
0
0
Camera calibration is a process of paramount importance in computer vision applications that require accurate quantitative measurements. The popular method developed by Zhang relies on the use of a large number of images of a planar grid of fiducial points captured in multiple poses. Although flexible and easy to implement, Zhang's method has some limitations. The simultaneous optimization of the entire parameter set, including the coefficients of a predefined distortion model, may result in poor distortion correction at the image boundaries or in miscalculation of the intrinsic parameters, even with a reasonably small reprojection error. Indeed, applications involving image stitching (e.g. multi-camera systems) require accurate mapping of distortion up to the outermost regions of the image. Moreover, intrinsic parameters affect the accuracy of camera pose estimation, which is fundamental for applications such as vision servoing in robot navigation and automated assembly. This paper proposes a method for estimating the complete set of calibration parameters from a single image of a planar speckle pattern covering the entire sensor. The correspondence between image points and physical points on the calibration target is obtained using Digital Image Correlation. The effective focal length and the extrinsic parameters are calculated separately after a prior evaluation of the principal point. At the end of the procedure, a dense and uniform model-free distortion map is obtained over the entire image. Synthetic data with different noise levels were used to test the feasibility of the proposed method and to compare its metrological performance with Zhang's method. Real-world tests demonstrate the potential of the developed method to reveal aspects of the image formation that are hidden by averaging over multiple images.
6/26/2024
Dusk Till Dawn: Self-supervised Nighttime Stereo Depth Estimation using Visual Foundation Models
Madhu Vankadari, Samuel Hodgson, Sangyun Shin, Kaichen Zhou Andrew Markham, Niki Trigoni
0
0
Self-supervised depth estimation algorithms rely heavily on frame-warping relationships, exhibiting substantial performance degradation when applied in challenging circumstances, such as low-visibility and nighttime scenarios with varying illumination conditions. Addressing this challenge, we introduce an algorithm designed to achieve accurate self-supervised stereo depth estimation focusing on nighttime conditions. Specifically, we use pretrained visual foundation models to extract generalised features across challenging scenes and present an efficient method for matching and integrating these features from stereo frames. Moreover, to prevent pixels violating photometric consistency assumption from negatively affecting the depth predictions, we propose a novel masking approach designed to filter out such pixels. Lastly, addressing weaknesses in the evaluation of current depth estimation algorithms, we present novel evaluation metrics. Our experiments, conducted on challenging datasets including Oxford RobotCar and Multi-Spectral Stereo, demonstrate the robust improvements realized by our approach. Code is available at: https://github.com/madhubabuv/dtd
5/21/2024
Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation
Ning-Hsu Wang, Yu-Lun Liu
0
0
Accurately estimating depth in 360-degree imagery is crucial for virtual reality, autonomous navigation, and immersive media applications. Existing depth estimation methods designed for perspective-view imagery fail when applied to 360-degree images due to different camera projections and distortions, whereas 360-degree methods perform inferior due to the lack of labeled data pairs. We propose a new depth estimation framework that utilizes unlabeled 360-degree data effectively. Our approach uses state-of-the-art perspective depth estimation models as teacher models to generate pseudo labels through a six-face cube projection technique, enabling efficient labeling of depth in 360-degree images. This method leverages the increasing availability of large datasets. Our approach includes two main stages: offline mask generation for invalid regions and an online semi-supervised joint training regime. We tested our approach on benchmark datasets such as Matterport3D and Stanford2D3D, showing significant improvements in depth estimation accuracy, particularly in zero-shot scenarios. Our proposed training pipeline can enhance any 360 monocular depth estimator and demonstrates effective knowledge transfer across different camera projections and data types. See our project page for results: https://albert100121.github.io/Depth-Anywhere/
6/19/2024
SDL-MVS: View Space and Depth Deformable Learning Paradigm for Multi-View Stereo Reconstruction in Remote Sensing
Yong-Qiang Mao, Hanbo Bi, Liangyu Xu, Kaiqiang Chen, Zhirui Wang, Xian Sun, Kun Fu
0
0
Research on multi-view stereo based on remote sensing images has promoted the development of large-scale urban 3D reconstruction. However, remote sensing multi-view image data suffers from the problems of occlusion and uneven brightness between views during acquisition, which leads to the problem of blurred details in depth estimation. To solve the above problem, we re-examine the deformable learning method in the Multi-View Stereo task and propose a novel paradigm based on view Space and Depth deformable Learning (SDL-MVS), aiming to learn deformable interactions of features in different view spaces and deformably model the depth ranges and intervals to enable high accurate depth estimation. Specifically, to solve the problem of view noise caused by occlusion and uneven brightness, we propose a Progressive Space deformable Sampling (PSS) mechanism, which performs deformable learning of sampling points in the 3D frustum space and the 2D image space in a progressive manner to embed source features to the reference feature adaptively. To further optimize the depth, we introduce Depth Hypothesis deformable Discretization (DHD), which achieves precise positioning of the depth prior by adaptively adjusting the depth range hypothesis and performing deformable discretization of the depth interval hypothesis. Finally, our SDL-MVS achieves explicit modeling of occlusion and uneven brightness faced in multi-view stereo through the deformable learning paradigm of view space and depth, achieving accurate multi-view depth estimation. Extensive experiments on LuoJia-MVS and WHU datasets show that our SDL-MVS reaches state-of-the-art performance. It is worth noting that our SDL-MVS achieves an MAE error of 0.086, an accuracy of 98.9% for <0.6m, and 98.9% for <3-interval on the LuoJia-MVS dataset under the premise of three views as input.
5/28/2024