Observabilty Properties and Deterministic Algorithms in Visual-Inertial Structure from Motion

Agostino Martinelli (1)

1 E-MOTION - Geometry and Probability for Motion and Action

This paper discusses the visual-inertial structure from motion problem (VI-SfM problem) with special focus on the following three fundamen- tal issues: observability properties, resolvability in closed form and data association. Regarding the rst issue, after a discussion about the cur- rent state of the art, the paper rst investigates more complex scenarios. Specically, with respect to the common formulation, which assumes three orthogonal accelerometers and three orthogonal gyroscopes, the analysis is extended to cope with the cases of a reduced number of inertial sensors and any number of point features observed by monoc- ular vision. In particular, the minimal case of a single accelerometer, no gyroscope and a single point feature is addressed. Additionally, the analysis accounts for biased measurements and unknown extrinsic cam- era calibration. The results derived for these new and very challenging scenarios have interesting consequences both from a technological and neuroscientic perspective. Regarding the second issue, a simple closed form solution to the VI-SfM is presented. This solution expresses the structure of the scene and the motion only in terms of the visual and in-ertial measurements collected during a short time interval. This allows introducing deterministic algorithms able to simultaneously determine the structure of the scene together with the motion without the need for any initialization or prior knowledge. Additionally, the closed-form so- lution allows us to identify the conditions under which the VI-SfM has a nite number of solutions. Specically, it is shown that the prob- lem can have a unique solution, two distinct solutions or innite so- lutions depending on the trajectory, on the number of point-features and on their arrangement in the 3D space and on the number of cam- era images. Finally, the paper discusses the third issue, i.e., the data association problem. Starting from basic results in computer vision, it is shown that, by exploiting the information provided by the inertial measurements, a single point correspondence (in the case of a planar motion) and two point correspondences (for a general 3D motion) are sucient to characterize the motion between two camera poses. This allows us to use a 1-point RANSAC algorithm (in the planar case) or a 2-point RANSAC algorithm (in the general 3D case) to detect out- liers. The paper concludes with some discussion about connections to related research elds both in the framework of computer science and neuroscience.

Domaines: Automatique / Robotique; Vision par ordinateur et reconnaissance de formes [cs.CV]

Agostino Martinelli. Observabilty Properties and Deterministic Algorithms in Visual-Inertial Structure from Motion. Foundations and Trends in Robotics, 2013, pp.1-75. ⟨hal-01096948⟩