CodaLab Competitions: An open source platform to organize scientific challenges

Adrien Pavao (1, 2) , Isabelle Guyon (1, 2) , Anne-Catherine Letournel (1, 2) , Xavier Baró (3) , Hugo Escalante (4) , Sergio Escalera (5) , Tyler Thomas (6) , Zhen Xu (7)

1 LISN - Laboratoire Interdisciplinaire des Sciences du Numérique
2 TAU - TAckling the Underspecified
3 UOC - Universitat Oberta de Catalunya [Barcelona]
4 INAOE - Instituto Nacional de Astrofísica, Óptica y Electrónica
5 UB - Universitat de Barcelona
6 Tier0 - Tier0 Software Development LLC
7 4Paradigm

CodaLab Competitions is an open source web platform designed to help data scientists and research teams to crowd-source the resolution of machine learning problems through the organization of competitions, also called challenges or contests. CodaLab Competitions provides useful features such as multiple phases, results and code submissions, multi-score leaderboards, and jobs running inside Docker containers. The platform is very flexible and can handle large scale experiments, by allowing organizers to upload large datasets and provide their own CPU or GPU compute workers.

Domaines: Intelligence artificielle [cs.AI]; Statistiques [math.ST]; Machine Learning [stat.ML]

Adrien Pavao, Isabelle Guyon, Anne-Catherine Letournel, Xavier Baró, Hugo Escalante, et al.. CodaLab Competitions: An open source platform to organize scientific challenges. [Technical Report] Université Paris-Saclay, FRA. 2022. ⟨hal-03629462⟩