Paraphrases do not explain word analogies

Louis Fournier (1) , Ewan Dunbar (1, 2)

1 CoML - Apprentissage machine et développement cognitif
2 University of Toronto

Many types of distributional word embeddings (weakly) encode linguistic regularities as directions (the difference between jump and jumped will be in a similar direction to that of walk and walked, and so on). Several attempts have been made to explain this fact. We respond to Allen and Hospedales' recent (ICML, 2019) theoretical explanation, which claims that word2vec and GloVe will encode linguistic regularities whenever a specific relation of paraphrase holds between the four words involved in the regularity. We demonstrate that the explanation does not go through: the paraphrase relations needed under this explanation do not hold empirically.

Domaines: Informatique [cs]

Louis Fournier, Ewan Dunbar. Paraphrases do not explain word analogies. EACL 2021 - 16th Conference of the European Chapter of the Association for Computational Linguistics, Apr 2021, Online, France. pp.2129-2134, ⟨10.18653/v1/2021.eacl-main.182⟩. ⟨hal-03697442⟩