A Markovian approach to distributional semantics with application to semantic compositionality

Edouard Grave (1, 2, 3) , Guillaume Obozinski (4, 5) , Francis Bach (2, 3)

1 LBNL - Lawrence Berkeley National Laboratory [Berkeley]
2 SIERRA - Statistical Machine Learning and Parsimony
3 LIENS - Laboratoire d'informatique de l'école normale supérieure
4 LIGM - Laboratoire d'Informatique Gaspard-Monge
5 imagine [Marne-la-Vallée]

In this article, we describe a new approach to distributional semantics. This approach relies on a generative model of sentences with latent variables, which takes the syntax into account by using syntactic dependency trees. Words are then represented as posterior distributions over those latent classes, and the model allows to naturally obtain in-context and out-of-context word representations, which are comparable. We train our model on a large corpus and demonstrate the compositionality capabilities of our approach on different datasets.

Mots clés: Word representation; Word embedding; Hidden markov model; Distributional semantics

Domaines: Informatique et langage [cs.CL]; Apprentissage [cs.LG]

Edouard Grave, Guillaume Obozinski, Francis Bach. A Markovian approach to distributional semantics with application to semantic compositionality. International Conference on Computational Linguistics (Coling), International Committee on Computational Linguistics (ICCL), Aug 2014, Dublin, Ireland. pp.1447 - 1456. ⟨hal-01080309⟩