Real-time tracking of 3D elastic objects with an RGB-D sensor

Antoine Petit (1, 2) , Vincenzo Lippiello (2) , Bruno Siciliano (2)

1 Inria Siège
2 DIETI - Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione [Napoli]

This paper presents a method to track in real-time a 3D textureless object which undergoes large deformations such as elastic ones, and rigid motions, using the point cloud data provided by an RGB-D sensor. This solution is expected to be useful for enhanced manipulation of humanoid robotic systems. Our framework relies on a prior visual segmentation of the object in the image. The segmented point cloud is registered first in a rigid manner and then by non-rigidly fitting the mesh, based on the Finite Element Method to model elasticity, and on geometrical point-to-point correspondences to compute external forces exerted on the mesh. The real-time performance of the system is demonstrated on synthetic and real data involving challenging deformations and motions.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Modélisation et simulation; Robotique [cs.RO]

Antoine Petit, Vincenzo Lippiello, Bruno Siciliano. Real-time tracking of 3D elastic objects with an RGB-D sensor. IROS 2015 - IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Sep 2015, Hamburg, Germany. pp.3914-3921, ⟨10.1109/IROS.2015.7353928⟩. ⟨hal-01617309⟩