Multi-Visual-Inertial System: Analysis, Calibration and Estimation

Overview

• The paper focuses on state estimation for multi-visual-inertial systems (MVIS), which involves fusing data from multiple asynchronous inertial measurement units (IMUs) or gyroscopes and global and/or rolling shutter cameras.
• The researchers developed sensor fusion algorithms to optimally combine these various sensor inputs, while also calibrating the associated visual-inertial sensors, including IMU/camera intrinsics and IMU-IMU(or camera) spatiotemporal extrinsics.
• They introduce a new analytical combined IMU integration method called ACI3 to preintegrate IMU measurements, which is used to fuse auxiliary IMUs and/or gyroscopes alongside a base IMU.
• The paper also includes observability analysis of MVIS, identifying the standard four unobservable directions as well as degenerate motions for IMU-IMU spatiotemporal extrinsics and auxiliary inertial intrinsics.

Plain English Explanation

The researchers in this paper looked at ways to combine information from multiple cameras and motion sensors (called inertial measurement units or IMUs) to get a better understanding of the position and orientation of a system. This is important for applications like robotics, augmented reality, and autonomous vehicles.

One key challenge is that the cameras and IMUs may not be perfectly synchronized or calibrated. So the researchers developed new algorithms to fuse the sensor data in an optimal way, while also calibrating the sensors to account for things like the camera's field of view and the exact positioning of the IMUs.

They introduced a new mathematical technique called ACI3 to help combine the measurements from multiple IMUs. And they analyzed the observability of the system - meaning they identified which properties of the system's state can be reliably estimated from the sensor data, and which ones are hard to pin down.

Overall, the goal was to create a robust and accurate way to track the motion and position of a system using a variety of visual and inertial sensors. This could have applications in areas like robotics and augmented reality.

Technical Explanation

The paper presents a sensor fusion framework for multi-visual-inertial systems (MVIS). The key contributions include:

1. Sensor Fusion: The researchers developed algorithms to optimally fuse data from an arbitrary number of asynchronous inertial measurement units (IMUs) or gyroscopes and global and/or rolling shutter cameras.

2. Sensor Calibration: They addressed the full calibration of the visual-inertial sensors, including the IMU or camera intrinsics and the IMU-IMU(or camera) spatiotemporal extrinsics, as well as the image readout time of rolling-shutter cameras.

3. ACI3 Preintegration: The team introduced a new analytic combined IMU integration method called ACI3 to preintegrate IMU measurements. This enables fusing auxiliary IMUs and/or gyroscopes alongside a base IMU.

4. Observability Analysis: The researchers performed an observability analysis of MVIS, proving that the standard four unobservable directions remain, regardless of the number of inertial sensors used. They also identified degenerate motions for IMU-IMU spatiotemporal extrinsics and auxiliary inertial intrinsics.

5. Experimental Validation: In addition to extensive simulations, the team built their own MVIS sensor rig and collected over 25 real-world datasets. They showed that their proposed calibration method achieves competing accuracy with improved convergence and repeatability compared to the state-of-the-art Kalibr calibration.

Critical Analysis

The paper presents a comprehensive approach to state estimation for multi-visual-inertial systems. The researchers have addressed several key challenges, including sensor synchronization, calibration, and observability analysis.

One potential limitation is the computational complexity of fusing data from multiple IMUs and cameras. The authors claim to have reduced this complexity by leveraging IMU-IMU rigid-body constraints, but the scalability of the approach for larger sensor arrays could still be an issue.

Additionally, the real-world experiments were conducted using the researchers' own custom MVIS sensor rig. It would be valuable to see the performance of their methods on a wider range of hardware setups, including commercial off-the-shelf components, to assess the generalizability of the results.

Further research could also explore the use of event-based cameras in MVIS, which could provide additional benefits in terms of temporal resolution and robustness to motion blur.

Conclusion

This paper presents a comprehensive framework for state estimation in multi-visual-inertial systems. The researchers developed novel sensor fusion algorithms and calibration methods to optimally combine data from multiple asynchronous IMUs and cameras. Their work includes important theoretical contributions, such as observability analysis, as well as experimental validation on real-world datasets.

The proposed techniques could have significant practical applications in fields like robotics, augmented reality, and autonomous vehicles, where accurate and robust motion tracking is crucial. By open-sourcing their work, the researchers aim to benefit the wider research community and drive further advancements in this area.