Arabic Statistical N-gram Models

Karima Meftouh (1) , Kamel Smaïli (2) , Mohamed Tayeb Laskri (1)

1 UBMA - Université Badji Mokhtar [Annaba]
2 PAROLE - Analysis, perception and recognition of speech

In this work we propose to investigate statistical language models for Arabic. Several experiments using different smoothing techniques have been carried out on a small corpus extracted from a daily newspaper. The sparseness data conducts us to investigate other solutions without increasing the size of the corpus. A word segmentation has been operated in order to increase the statistical viability of the corpus. This leads to a better performance in terms of normalized perplexity

Mots clés: Morpheme-based n-gram models; word-based n-gram models; Arabic; N-gram models; Statistical language model; Perplexity

Domaines: Informatique et langage [cs.CL]

Karima Meftouh, Kamel Smaïli, Mohamed Tayeb Laskri. Arabic Statistical N-gram Models. International Review on Computers and Software (IRECOS), 2009, 4 (1). ⟨hal-01639807⟩