JointLoc: A Real-time Visual Localization Framework for Planetary UAVs Based on Joint Relative and Absolute Pose Estimation
0
🗣️
Sign in to get full access
Overview
- Proposed method called JointLoc for accurately estimating the real-time position of unmanned aerial vehicles (UAVs) in planetary environments without relying on global navigation satellite systems (GNSS)
- Addresses challenges of visual localization in planetary scenes due to lack of landmarks and differences between satellite maps and UAV images
- JointLoc fuses absolute 2-degree-of-freedom (2-DoF) pose and relative 6-degree-of-freedom (6-DoF) pose to estimate UAV position in world coordinates
- Evaluated on a proposed planetary UAV image dataset with simulated and real Martian topography
Plain English Explanation
Navigating UAVs in planetary environments like Mars can be challenging because the terrain often lacks clear landmarks that the UAV's cameras can use to determine its exact location. Additionally, the satellite maps used for navigation may look quite different from the actual images captured by the UAV's cameras, making it hard to match them up.
To address this, the researchers developed a system called JointLoc. JointLoc combines two types of position information to estimate the UAV's location in the world coordinate system:
- Absolute 2-DoF pose: This provides a rough estimate of the UAV's position and orientation in a 2D plane.
- Relative 6-DoF pose: This tracks the UAV's position and orientation in 3D space relative to its previous location.
By adaptively fusing these two position cues, JointLoc can accurately determine the UAV's real-time position without relying on GNSS, which may not be available in planetary environments.
The researchers evaluated JointLoc on a new dataset of simulated and real Martian terrain. Compared to other SLAM (simultaneous localization and mapping) systems like ORB-SLAM2 and ORB-SLAM3, JointLoc achieved significantly lower error in estimating the UAV's trajectory, with a root-mean-square error of just 0.237 meters over paths up to 1,000 meters long.
Technical Explanation
The key innovation of JointLoc is its adaptive fusion of absolute 2-DoF and relative 6-DoF pose estimates to accurately localize the UAV in planetary environments without GNSS.
The absolute 2-DoF pose is obtained by matching the UAV's camera images to satellite maps of the planetary surface using edge detection neural networks. This provides a rough estimate of the UAV's position and orientation in a 2D plane.
The relative 6-DoF pose is estimated using visual SLAM techniques that track the UAV's motion through the 3D environment by analyzing the camera images. This provides a more detailed but drifting estimate of the UAV's position and orientation in 3D space.
JointLoc adaptively combines these two pose estimates, relying more on the absolute 2-DoF pose when the relative 6-DoF pose is drifting, and vice versa. This allows it to accurately localize the UAV without relying solely on the potentially noisy satellite map matching.
The researchers evaluated JointLoc on a new planetary UAV image dataset that includes both simulated Martian environments as well as real images from the Ingenuity helicopter on Mars. Compared to state-of-the-art SLAM systems, JointLoc achieved significantly lower position estimation error on this challenging dataset.
Critical Analysis
The researchers have addressed an important challenge in planetary UAV navigation by developing JointLoc, a system that can accurately localize UAVs without access to GNSS. The adaptive fusion of absolute and relative pose estimates is a clever approach to overcome the limitations of each individual technique.
However, the paper does not discuss some potential limitations of the proposed method. For example, the accuracy of the absolute 2-DoF pose estimates may degrade in areas with less distinctive visual features, and the relative 6-DoF pose can still drift over long trajectories. Further research is needed on improving the robustness and long-term stability of the localization system.
Additionally, the evaluation was limited to a single dataset, and it would be valuable to test JointLoc on a wider range of planetary environments to better understand its generalization capabilities. Comparisons to other emerging localization techniques, such as those using ground-to-aerial image matching, could also provide additional insight.
Overall, JointLoc represents an important step forward in enabling robust and accurate visual localization for planetary UAVs, but there are still opportunities to further refine and expand the approach.
Conclusion
The JointLoc system proposed in this paper addresses a crucial challenge in enabling unmanned aerial vehicles to navigate effectively in planetary environments without access to global navigation satellite systems. By adaptively fusing absolute and relative pose estimates, JointLoc can accurately localize UAVs in real-time, even in scenes that lack clear visual landmarks.
The evaluation results on a new planetary UAV dataset demonstrate the significant performance advantages of JointLoc over existing SLAM-based approaches. This work represents an important advancement towards making UAV-based exploration and surveying of other planets more reliable and practical.
While the paper identifies some promising directions for further research, the core JointLoc approach serves as a valuable foundation for continued development of robust and accurate visual localization systems for planetary UAVs.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🗣️
0
JointLoc: A Real-time Visual Localization Framework for Planetary UAVs Based on Joint Relative and Absolute Pose Estimation
Xubo Luo, Xue Wan, Yixing Gao, Yaolin Tian, Wei Zhang, Leizheng Shu
Unmanned aerial vehicles (UAVs) visual localization in planetary aims to estimate the absolute pose of the UAV in the world coordinate system through satellite maps and images captured by on-board cameras. However, since planetary scenes often lack significant landmarks and there are modal differences between satellite maps and UAV images, the accuracy and real-time performance of UAV positioning will be reduced. In order to accurately determine the position of the UAV in a planetary scene in the absence of the global navigation satellite system (GNSS), this paper proposes JointLoc, which estimates the real-time UAV position in the world coordinate system by adaptively fusing the absolute 2-degree-of-freedom (2-DoF) pose and the relative 6-degree-of-freedom (6-DoF) pose. Extensive comparative experiments were conducted on a proposed planetary UAV image cross-modal localization dataset, which contains three types of typical Martian topography generated via a simulation engine as well as real Martian UAV images from the Ingenuity helicopter. JointLoc achieved a root-mean-square error of 0.237m in the trajectories of up to 1,000m, compared to 0.594m and 0.557m for ORB-SLAM2 and ORB-SLAM3 respectively. The source code will be available at https://github.com/LuoXubo/JointLoc.
Read more5/14/2024
0
UAV-VisLoc: A Large-scale Dataset for UAV Visual Localization
Wenjia Xu, Yaxuan Yao, Jiaqi Cao, Zhiwei Wei, Chunbo Liu, Jiuniu Wang, Mugen Peng
The application of unmanned aerial vehicles (UAV) has been widely extended recently. It is crucial to ensure accurate latitude and longitude coordinates for UAVs, especially when the global navigation satellite systems (GNSS) are disrupted and unreliable. Existing visual localization methods achieve autonomous visual localization without error accumulation by matching the ground-down view image of UAV with the ortho satellite maps. However, collecting UAV ground-down view images across diverse locations is costly, leading to a scarcity of large-scale datasets for real-world scenarios. Existing datasets for UAV visual localization are often limited to small geographic areas or are focused only on urban regions with distinct textures. To address this, we define the UAV visual localization task by determining the UAV's real position coordinates on a large-scale satellite map based on the captured ground-down view. In this paper, we present a large-scale dataset, UAV-VisLoc, to facilitate the UAV visual localization task. This dataset comprises images from diverse drones across 11 locations in China, capturing a range of topographical features. The dataset features images from fixed-wing drones and multi-terrain drones, captured at different altitudes and orientations. Our dataset includes 6,742 drone images and 11 satellite maps, with metadata such as latitude, longitude, altitude, and capture date. Our dataset is tailored to support both the training and testing of models by providing a diverse and extensive data.
Read more5/21/2024
0
LVCP: LiDAR-Vision Tightly Coupled Collaborative Real-time Relative Positioning
Zhuozhu Jian, Qixuan Li, Shengtao Zheng, Xueqian Wang, Xinlei Chen
In air-ground collaboration scenarios without GPS and prior maps, the relative positioning of drones and unmanned ground vehicles (UGVs) has always been a challenge. For a drone equipped with monocular camera and an UGV equipped with LiDAR as an external sensor, we propose a robust and real-time relative pose estimation method (LVCP) based on the tight coupling of vision and LiDAR point cloud information, which does not require prior information such as maps or precise initial poses. Given that large-scale point clouds generated by 3D sensors has more accurate spatial geometric information than the feature point cloud generated by image, we utilize LiDAR point clouds to correct the drift in visual-inertial odometry (VIO) when the camera undergoes significant shaking or the IMU has a low signal-to-noise ratio. To achieve this, we propose a novel coarse-to-fine framework for LiDAR-vision collaborative localization. In this framework, we construct point-plane association based on spatial geometric information, and innovatively construct a point-aided Bundle Adjustment (BA) problem as the backend to simultaneously estimate the relative pose of the camera and LiDAR and correct the VIO drift. In this process, we propose a particle swarm optimization (PSO) based sampling algorithm to complete the coarse estimation of the current camera-LiDAR pose. In this process, the initial pose of the camera used for sampling is obtained based on VIO propagation, and the valid feature-plane association number (VFPN) is used to trigger PSO-sampling process. Additionally, we propose a method that combines Structure from Motion (SFM) and multi-level sampling to initialize the algorithm, addressing the challenge of lacking initial values.
Read more7/16/2024
0
Fusion of Visual-Inertial Odometry with LiDAR Relative Localization for Cooperative Guidance of a Micro-Scale Aerial Vehicle
V'aclav Pritzl, Matouv{s} Vrba, Petr v{S}tv{e}p'an, Martin Saska
A novel relative localization approach for guidance of a micro-scale UAV by a well-equipped aerial robot fusing VIO with LiDAR is proposed in this paper. LiDAR-based localization is accurate and robust to challenging environmental conditions, but 3D LiDARs are relatively heavy and require large UAV platforms, in contrast to lightweight cameras. However, visual-based self-localization methods exhibit lower accuracy and can suffer from significant drift with respect to the global reference frame. To benefit from both sensory modalities, we focus on cooperative navigation in a heterogeneous team of a primary LiDAR-equipped UAV and a secondary micro-scale camera-equipped UAV. We propose a novel cooperative approach combining LiDAR relative localization data with VIO output on board the primary UAV to obtain an accurate pose of the secondary UAV. The pose estimate is used to precisely and reliably guide the secondary UAV along trajectories defined in the primary UAV reference frame. The experimental evaluation has shown the superior accuracy of our method to the raw VIO output and demonstrated its capability to guide the secondary UAV along desired trajectories while mitigating VIO drift. Thus, such a heterogeneous system can explore large areas with LiDAR precision, as well as visit locations inaccessible to the large LiDAR-carrying UAV platforms, as was showcased in a real-world cooperative mapping scenario.
Read more8/27/2024