3D Object Reconstruction with Heterogeneous Sensor Data

Li Guan (1, 2) , Jean-Sébastien Franco (3, 4) , Marc Pollefeys (1, 2)

1 UNC - University of North Carolina [Chapel Hill]
2 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]
3 IPARLA - Visualization and manipulation of complex data on wireless mobile devices
4 LaBRI - Laboratoire Bordelais de Recherche en Informatique

In this paper, we reconstruct 3D objects with a heterogeneous sensor network of Range Imaging(RIM) sensors and high-res camcorders. With this setup, we first carry out simple but effective depth calibration for the RIM cameras. We then combine the camcorder silhouette cues and RIM camera depth information, for the reconstruction. Our main contribution is the proposal of a sensor fusion framework so that the computation is general, simple and scalable. Although we only discuss the camcorders and RIM cameras in this paper, the proposed framework can be applied to any type of vision sensors. It uses a space occupancy grid as a probabilistic 3D representation of scene contents. After defining sensing models for each type of sensors, the reconstruction is simply a Bayesian inference problem, and can be solved robustly. The experiments show that the recover full 3D closed shapes substantially improved the quality of the noisy RIM sensor measurement.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Li Guan, Jean-Sébastien Franco, Marc Pollefeys. 3D Object Reconstruction with Heterogeneous Sensor Data. International Symposium on 3D Data Processing, Visualization and Transmission, Jun 2008, Atlanta, United States. ⟨inria-00349099⟩