The Zero Resource Speech Challenge 2021: Spoken language modelling

Ewan Dunbar (1) , Mathieu Bernard (2, 3) , Nicolas Hamilakis (2, 3) , Tu Anh Nguyen (2, 3, 4) , Maureen de Seyssel (2, 3) , Patricia Rozé (2, 3) , Morgane Rivière (4) , Eugene Kharitonov (4) , Emmanuel Dupoux (2, 3)

1 University of Toronto
2 LSCP - Laboratoire de sciences cognitives et psycholinguistique
3 CoML - Apprentissage machine et développement cognitif
4 FAIR - Facebook AI Research [Paris]

We present the Zero Resource Speech Challenge 2021, which asks participants to learn a language model directly from audio, without any text or labels. The challenge is based on the Libri-light dataset, which provides up to 60k hours of audio from English audio books without any associated text. We provide a pipeline baseline system consisting on an encoder based on contrastive predictive coding (CPC), a quantizer ($k$-means) and a standard language model (BERT or LSTM). The metrics evaluate the learned representations at the acoustic (ABX discrimination), lexical (spot-the-word), syntactic (acceptability judgment) and semantic levels (similarity judgment). We present an overview of the eight submitted systems from four groups and discuss the main results.

Mots clés: Unsupervised speech; Lowresource; Language modelling; Zero-resource; Cognitive benchmarks

Domaines: Informatique et langage [cs.CL]; Intelligence artificielle [cs.AI]

Ewan Dunbar, Mathieu Bernard, Nicolas Hamilakis, Tu Anh Nguyen, Maureen de Seyssel, et al.. The Zero Resource Speech Challenge 2021: Spoken language modelling. Interspeech 2021 - Conference of the International Speech Communication Association, Aug 2021, Brno, Czech Republic. ⟨10.1109/TPAMI.2021.3083839⟩. ⟨hal-03329301v2⟩