Predicting Tongue Positions from Acoustics and Facial Features

Asterios Toutios (1) , Slim Ouni (2, 1)

1 PAROLE - Analysis, perception and recognition of speech
2 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication

We test the hypothesis that adding information regarding the positions of electromagnetic articulograph (EMA) sensors on the lips and jaw can improve the results of a typical acoustic-to-EMA mapping system, based on support vector regression, that targets the tongue sensors. Our initial motivation is to use such a system in the context of adding a tongue animation to a talking head built on the basis of concatenating bimodal acoustic-visual units. For completeness, we also train a system that maps only jaw and lip information to tongue information.

Domaines: Intelligence artificielle [cs.AI]

Asterios Toutios, Slim Ouni. Predicting Tongue Positions from Acoustics and Facial Features. 12th Annual Conference of the International Speech Communication Association - Interspeech 2011, Aug 2011, Florence, Italy. ⟨inria-00602412⟩