View synthesis for pose computation

Pierre Rolin (1) , Marie-Odile Berger (1) , Frédéric Sur (1)

1 MAGRIT-POST - Augmentation visuelle d'environnements complexes

Geometrical registration of a query image with respect to a 3D model, or pose estimation, is the cornerstone of many computer vision applications. It is often based on the matching of local photometric descriptors invariant to limited viewpoint changes. However, when the query image has been acquired from a camera position not covered by the model images, pose estimation is often not accurate and sometimes even fails, precisely because of the limited invariance of descriptors. In this paper, we propose to add descriptors to the model, obtained from synthesized views associated with virtual cameras completing the covering of the scene by the real cameras. We propose an efficient strategy to localize the virtual cameras in the scene and generate valuable descriptors from synthetic views. We also discuss a guided sampling strategy for registration in this context. Experiments show that the accuracy of pose estimation is dramatically improved when large viewpoint changes makes the matching of classic descriptors a challenging task.

Mots clés: pose estimation; view synthesis; feature matching; RANSAC

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Pierre Rolin, Marie-Odile Berger, Frédéric Sur. View synthesis for pose computation. Machine Vision and Applications, 2019, 30 (7-8), pp.1209-1227. ⟨10.1007/s00138-019-01045-5⟩. ⟨hal-02279616⟩