ChaLearn Looking at People RGB-D Isolated and Continuous Datasets for Gesture Recognition

Jun Wan (1) , Yibing Zhao (2) , Shuai Zhou (2) , Isabelle Guyon (3, 4, 5) , Sergio Escalera (6) , Stan Z. Li (1)

1 NLPR - National Laboratory of Pattern Recognition [Beijing]
2 MUST - Macau University of Science and Technology
3 TAO - Machine Learning and Optimisation
4 Chalearn
5 UP11 UFR Sciences - Université Paris-Sud - Paris 11 - Faculté des Sciences
6 CVC - Computer Vision Center (Centre de visio per computador)

In this paper, we present two large video multi-modal datasets for RGB and RGB-D gesture recognition: the ChaLearn LAP RGB-D Isolated Gesture Dataset (IsoGD) and the Continuous Gesture Dataset (ConGD). Both datasets are derived from the ChaLearn Gesture Dataset (CGD) that has a total of more than 50000 gestures for the "one-shot-learning" competition. To increase the potential of the old dataset, we designed new well curated datasets composed of 249 gesture labels, and including 47933 gestures manually labeled the begin and end frames in sequences. Using these datasets we will open two competitions on the CodaLab platform so that researchers can test and compare their methods for "user independent" gesture recognition. The first challenge is designed for gesture spotting and recognition in continuous sequences of gestures while the second one is designed for gesture classification from segmented data. The baseline method based on the bag of visual words (BoVW) model is also presented.

Domaines: Apprentissage [cs.LG]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Traitement du signal et de l'image [eess.SP]

Jun Wan, Yibing Zhao, Shuai Zhou, Isabelle Guyon, Sergio Escalera, et al.. ChaLearn Looking at People RGB-D Isolated and Continuous Datasets for Gesture Recognition. CVPR 2016 - IEEE Conference on Computer Vision and Pattern Recognition - Workshops, 2016, Las Vegas, United States. ⟨hal-01381151⟩