WEB-based listening test system for speech synthesis and speech conversion evaluation

Laurent Blin (1) , Olivier Boëffard (1) , Vincent Barreaud (1)

1 CORDIAL - Human-machine spoken dialogue

In this article, we propose a web based listening test system that can be used with a large range of listeners. Our main goals were to make the configuration of the tests as simple and flexible as possible, to simplify the recruiting of the testees and, of course, to keep track of the results using a relational database. This first version of our system can perform the most widely used listening tests in the speech processing community (AB-BA, ABX and MOS tests). It can also easily evolve and propose other tests implemented by the tester by means of a module interface. This scenario is explored in this article which proposes an implementation of a module for Comparison Mean Opinion Score (CMOS) tests and conduct of such an experiment. This test allowed us to extract from the BREF120 corpus a couple of voices of distinct supra-segmental characteristics. This system is offered to the speech synthesis and speech conversion community under free license.

Domaines: Apprentissage [cs.LG]; Interface homme-machine [cs.HC]; Son [cs.SD]; Intelligence artificielle [cs.AI]

Laurent Blin, Olivier Boëffard, Vincent Barreaud. WEB-based listening test system for speech synthesis and speech conversion evaluation. Language Resources and Evaluation Conference, May 2008, Marrakech, Morocco. ⟨hal-02012732⟩