The Signal Separation Evaluation Campaign (2007-2010): Achievements and Remaining Challenges

Emmanuel Vincent (1) , Shoko Araki (2) , Fabian J. Theis (3) , Guido Nolte (4) , Pau Bofill (5) , Hiroshi Sawada (2) , Alexey Ozerov (1) , B. Vikrham Gowreesunker (6) , Dominik Lutter (7) , Ngoc Q. K. Duong (1)

1 METISS - Speech and sound data modeling and processing
2 NTT Communication Science Laboratories
3 Institute of Bioinformatics and Systems Biology [München]
4 Fraunhofer FIRST.IDA - Department Intelligent Data Analysis [Berlin]
5 DAC-UPC - Department of Computer Architecture [Barcelona]
6 Texas Instruments - R&D center [USA]
7 IBB - Institute of Biomathematics and Biometry - Helmholtz Zentrum München

We present the outcomes of three recent evaluation campaigns in the field of audio and biomedical source separation. These campaigns have witnessed a boom in the range of applications of source separation systems in the last few years, as shown by the increasing number of datasets from 1 to 9 and the increasing number of submissions from 15 to 34. We first discuss their impact on the definition of a reference evaluation methodology, together with shared datasets and software. We then present the key results obtained over almost all datasets. We conclude by proposing directions for future research and evaluation, based in particular on the ideas raised during the related panel discussion at the Ninth International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA 2010).

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Emmanuel Vincent, Shoko Araki, Fabian J. Theis, Guido Nolte, Pau Bofill, et al.. The Signal Separation Evaluation Campaign (2007-2010): Achievements and Remaining Challenges. Signal Processing, 2012, 92, pp.1928-1936. ⟨10.1016/j.sigpro.2011.10.007⟩. ⟨inria-00630985⟩