Bayesian nonparametric inference for discovery probabilities: credible intervals and large sample asymptotics

Julyan Arbel (1, 2) , Stefano Favaro (2, 3) , Bernardo Nipoti (2, 3) , Yee Whye Teh (4)

1 MISTIS - Modelling and Inference of Complex and Structured Stochastic Systems
2 Collegio Carlo Alberto
3 UNITO - Università degli studi di Torino = University of Turin
4 Department of Statistics [Oxford]

Given a sample of size n from a population of individual belonging to different species with unknown proportions, a popular problem of practical interest consists in making inference on the probability Dn(l) that the (n+1)-th draw coincides with a species with frequency l in the sample, for any l=0,1,…,n. This paper contributes to the methodology of Bayesian nonparametric inference for Dn(l). Specifically, under the general framework of Gibbs-type priors we show how to derive credible intervals for the Bayesian nonparametric estimator of Dn(l), and we investigate the large n asymptotic behaviour of such an estimator. Of particular interest are special cases of our results obtained under the assumption of the two parameter Poisson-Dirichlet prior and the normalized generalized Gamma prior, which are two of the most commonly used Gibbs-type priors. With respect to these two prior assumptions, the proposed results are illustrated through a simulation study and a benchmark Expressed Sequence Tags dataset. To the best our knowledge, this illustration provides the first comparative study between the two parameter Poisson-Dirichlet prior and the normalized generalized Gamma prior in the context of Bayesian nonparemetric inference for Dn(l)

Mots clés: Good–Turing estimator; Bayesian nonparametrics; Asymptotics; credible intervals; discovery probability; Gibbs-type priors; normalized generalized Gamma prior; smoothing technique; two parameter Poisson-Dirichlet prior.

Domaines: Méthodologie [stat.ME]; Applications [stat.AP]; Théorie [stat.TH]; Calcul [stat.CO]

Julyan Arbel, Stefano Favaro, Bernardo Nipoti, Yee Whye Teh. Bayesian nonparametric inference for discovery probabilities: credible intervals and large sample asymptotics. Statistica Sinica, 2017, 27, pp.839-858. ⟨10.5705/ss.202015.0250⟩. ⟨hal-01203324⟩