Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora

Fabrice Lefèvre (1) , Djamel Mostefa (2) , Laurent Besacier (3) , Yannick Estève (4) , Matthieu Quignard (5) , Nathalie Camelin (4) , Benoit Favre (6, 7) , Bassam Jabaian (1, 8) , Lina Maria Rojas Barahona (5)

1 LIA - Laboratoire Informatique d'Avignon
2 ELDA - Evaluations and Language resources Distribution Agency
3 GETALP - Groupe d’Étude en Traduction Automatique/Traitement Automatisé des Langues et de la Parole
4 LIUM - Laboratoire d'Informatique de l'Université du Mans
5 SYNALP - Natural Language Processing : representations, inference and semantics
6 LIF - Laboratoire d'informatique Fondamentale de Marseille - UMR 6166
7 TALEP - Traitement Automatique du Langage Ecrit et Parlé
8 LIG - Laboratoire d'Informatique de Grenoble

The PORTMEDIA project is intended to develop new corpora for the evaluation of spoken language understanding systems. The newly collected data are in the field of human-machine dialogue systems for tourist information in French in line with the MEDIA corpus. Transcriptions and semantic annotations, obtained by low-cost procedures, are provided to allow a thorough evaluation of the systems' capabilities in terms of robustness and portability across languages and domains. A new test set with some adaptation data is prepared for each case: in Italian as an example of a new language, for ticket reservation as an example of a new domain. Finally the work is complemented by the proposition of a new high level semantic annotation scheme well-suited to dialogue data.

Mots clés: spoken language understanding system; human-machine dialogue corpus; portability

Domaines: Informatique et langage [cs.CL]

Fabrice Lefèvre, Djamel Mostefa, Laurent Besacier, Yannick Estève, Matthieu Quignard, et al.. Leveraging study of robustness and portability of spoken language understanding systems across languages and domains: the PORTMEDIA corpora. The International Conference on Language Resources and Evaluation, May 2012, Istanbul, Turkey. ⟨hal-00683433⟩