Using Curvilinear Features in Focus for Registering a Single Image to a 3D Object

Hatem A. Rashwan (1) , Sylvie Chambon (1, 2) , Pierre Gurdjos (1, 3) , Géraldine Morin (1, 2) , Vincent Charvillat (1, 2)

1 IRIT-REVA - Real Expression Artificial Life
2 Toulouse INP - Institut National Polytechnique (Toulouse)
3 CNRS - Centre National de la Recherche Scientifique

In the context of 2D/3D registration, this paper introduces an approach that allows for matching features detected in two different modalities, photographs, and 3D models, by using a common 2D representation. More precisely, 2D images are matched with a set of depth images representing the 3D model. After introducing the concept of Curvilinear Saliency, which is related to curvature estimation, we propose a new ridge and valley detector for depth images rendered from 3D models. A variant of this detector is adapted to photographs, first by considering multi-scale features and second by integrating the focus curve principle. Finally, a registration algorithm determines the correct view of the 3D model and, thus, the pose of the photograph. This approach relies on the Histogram of Curvilinear Saliency (HCS), an adaptation of the Histogram of Oriented Gradients (HOG) to the proposed features in 2D and 3D. The presented results highlight both the quality of the features detected in terms of repeatability and the interest of the approach for registration and pose estimation.

Mots clés: 2D-3D registration; Pose estimation; Object detection; Feature extraction; Curvilinear saliency

Domaines: Traitement des images [eess.IV]

Hatem A. Rashwan, Sylvie Chambon, Pierre Gurdjos, Géraldine Morin, Vincent Charvillat. Using Curvilinear Features in Focus for Registering a Single Image to a 3D Object. IEEE Transactions on Image Processing, 2019, 28 (9), pp.4429- 4443. ⟨10.1109/TIP.2019.2911484⟩. ⟨hal-02891651⟩