Performance Analysis of Information Retrieval Systems

Juli Ayter (1) , Cecile Desclaux (1) , Adrian-Gabriel Chifu (2) , Sébastien Dejean (3) , Josiane Mothe (2)

1 INSA Toulouse - Institut National des Sciences Appliquées - Toulouse
2 IRIT-SIG - Systèmes d’Informations Généralisées
3 IMT - Institut de Mathématiques de Toulouse UMR5219

It has been shown that there is not a best information retrieval system configuration which would work for any query, but rather that performance can vary from one query to another. It would be interesting if a meta-system could decide which system should process a new query by learning from the context of previously submitted queries. This paper reports a deep analysis considering more than 80,000 search engine configurations applied to 100 queries and the corresponding performance. The goal of the analysis is to identify which search engine configuration responds best to a certain type of query. We considered two approaches to define query types: one is based on query clustering according to the query performance (their difficulty), while the other approach uses various query features (including query difficulty predictors) to cluster queries. We identified two parameters that should be optimized first. An important outcome is that we could not obtain strong conclusive results; considering the large number of systems and methods we used, this result could lead to the conclusion that current query features does not fit the optimizing problem.

Mots clés: Classification; Optimization; Random forest; Adaptive information retrieval; Information retrieval; Query difficulty

Domaines: Théorie de l'information [cs.IT]; Recherche d'information [cs.IR]

Juli Ayter, Cecile Desclaux, Adrian-Gabriel Chifu, Sébastien Dejean, Josiane Mothe. Performance Analysis of Information Retrieval Systems. 3rd Spanish Conference on Information Retrieval (CERI 2014), Jun 2014, Coruna, Spain. pp.1-12. ⟨hal-01119086⟩