Spatio-Temporal Human Shape Completion With Implicit Function Networks

Boyao Zhou (1, 2) , Jean-Sébastien Franco (1) , Federica Bogo (3) , Edmond Boyer (1)

1 MORPHEO - Capture and Analysis of Shapes in Motion
2 MSR - INRIA - Microsoft Research - Inria Joint Centre
3 MSR - Microsoft Research

We address the problem of inferring a human shape from partial observations, such as depth images, in temporal sequences. Deep Neural Networks (DNN) have been shown successful to estimate detailed shapes on a frame-by-frame basis but consider yet little or no temporal information over frame sequences for detailed shape estimation. Recently, networks that implicitly encode shape occupancy using MLP layers have shown very promising results for such single-frame shape inference, with the advantage of reducing the dimensionality of the problem and providing continuously encoded results. In this work we propose to generalize implicit encoding to spatio-temporal shape inference with spatio-temporal implicit function networks or STIF-Nets, where temporal redundancy and continuity is expected to improve the shape and motion quality. To validate these added benefits, we collect and train with motion data from CAPE for dressed humans, and DFAUST for body shapes with no clothing. We show our model's ability to estimate shapes for a set of input frames, and interpolate between them. Our results show that our method outperforms existing state of the art methods, in particular the single-frame methods for detailed shape estimation.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Boyao Zhou, Jean-Sébastien Franco, Federica Bogo, Edmond Boyer. Spatio-Temporal Human Shape Completion With Implicit Function Networks. International Conference on 3D Vision, Dec 2021, Online, United Kingdom. ⟨hal-03381387⟩