A new penalized criterion for variable selection and clustering using genotypic data

Dominique Bontemps (1) , Wilson Toussile (1)

1 LM-Orsay - Laboratoire de Mathématiques d'Orsay

We consider the problem of estimating the number of components and the relevant variables in a mixture model for multilocus genotypic data. A new penalized maximum likelihood criterion is proposed, and a non-asymptotic oracle inequality is obtained. Further, under weak assumptions on the true probability underlying the observations, the selected model is asymptotically consistent. On a practical aspect, the shape of our proposed penalty function is defined up to a multiplicative constant which is calibrated thanks to the slope heuristics, in an automatic data-driven procedure. Using simulated data, we found that this procedure improves the performances of the selection procedure with respect to classical criteria such as BIC and AIC. The new criterion gives an answer to the question ``Which criterion for which sample size?''.

Mots clés: Variables selection; Penalized Likelihood; Slope heuristics; Mixture multinomial models; Population genetics

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Dominique Bontemps, Wilson Toussile. A new penalized criterion for variable selection and clustering using genotypic data. 2010. ⟨hal-00453236v1⟩