Motion synthesis and editing for the generation of new sign language content

Lucie Naert (1) , Caroline Larboulette (1) , Sylvie Gibet (1)

1 EXPRESSION - Expressiveness in Human Centered Data/Media

Existing work on the animation of signing avatars often relies on pure procedural techniques or on the playback of Motion Capture (MoCap) data. While the first solution results in robotic and unnatural motions, the second one is very limited in the number of signs that it can produce. In this paper, we propose to implement data-driven motion synthesis techniques to increase the variety of Sign Language (SL) motions that can be made from a limited database. In order to generate new signs and inflection mechanisms based on an annotated French Sign Language MoCap corpus, we rely on phonological recombination, i.e. on the motion retrieval and modular reconstruction of SL content at a phonological level with a particular focus on three phonological components of SL: hand placement, hand configuration and hand movement. We propose to modify the values taken by those components in different signs to create their inflected version or completely new signs by (i) applying motion retrieval at a phonological level to exchange the value of one component without any modification, (ii) editing the retrieved data with different operators, or, (iii) using conventional motion generation techniques such as interpolation or inverse kinematics, which are parameterized to comply to the kinematic properties of real motion observed in the data set. The quality of the synthesized motions is perceptually assessed through two distinct evaluations that involved 75 and 53 participants respectively.

Mots clés: Sign Language; Motion Synthesis; Motion Capture; Phonological Combination; Avatar

Domaines: Informatique

Lucie Naert, Caroline Larboulette, Sylvie Gibet. Motion synthesis and editing for the generation of new sign language content. Machine Translation, 2021, 35 (3), pp.405-430. ⟨10.1007/s10590-021-09268-y⟩. ⟨hal-03324348⟩