Layer adaptation for transfer of expressivity in speech synthesis

Ajinkya Kulkarni (1) , Vincent Colotte (1) , Denis Jouvet (1)

1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication

Expressive speech synthesis using parametric approaches is constrained by the style of the speech corpus used. In this paper, we present the development of an expressive speech synthesis for a new speaker voice without requiring a specific recording of expressive speech by new speaker. We propose deep neural network based layer adaptation framework for transferring the expressive characteristics of speech to a new speaker's voice for which only neutral speech data is available. The focus of the work is on investigating transfer learning mechanism, which will accelerate the efforts towards exploiting existing expressive speech corpus. Experiments using expressive Caroline speech corpus and neutral Lisa speech corpus shows layer adaptation technique is able to transfer expressive characteristics while keeping the speaker's style characteristics.

Mots clés: expressive speech synthesis; deep learning; transfer learning; domain adaptation; emotion

Domaines: Intelligence artificielle [cs.AI]; Traitement du signal et de l'image [eess.SP]

Ajinkya Kulkarni, Vincent Colotte, Denis Jouvet. Layer adaptation for transfer of expressivity in speech synthesis. LTC'19 - 9th Language & Technology Conference, May 2019, Poznan, Poland. ⟨hal-02177945⟩