Per Channel Automatic Annotation of Sign Language Motion Capture Data

Lucie Naert (1) , Clément Reverdy (1) , Caroline Larboulette (1) , Sylvie Gibet (1)

1 EXPRESSION - Expressiveness in Human Centered Data/Media

Manual annotation is an expensive and time consuming task partly due to the high number of linguistic channels that usually compose sign language data. In this paper, we propose to automatize the annotation of sign language motion capture data by processing each channel separately. Motion features (such as distances between joints or facial descriptors) that take advantage of the 3D nature of motion capture data and the specificity of the channel are computed in order to (i) segment and (ii) label the sign language data. Two methods of automatic annotation of French Sign Language utterances using similar processes are developed. The first one describes the automatic annotation of thirty-two hand configurations while the second method describes the annotation of facial expressions using a closed vocabulary of seven expressions. Results for the two methods are then presented and discussed.

Mots clés: automatic annotation; automatic segmentation; motion capture; French Sign Language; linguistic channels

Domaines: Informatique

Lucie Naert, Clément Reverdy, Caroline Larboulette, Sylvie Gibet. Per Channel Automatic Annotation of Sign Language Motion Capture Data. Workshop on the Representation and Processing of Sign Languages: Involving the Language Community, LREC 2018, May 2018, Miyazaki Japan. ⟨hal-01851404⟩