A review of weighting schemes for bag of visual words image retrieval

Pierre Tirilly (1) , Vincent Claveau (1) , Patrick Gros (1)

1 TEXMEX - Multimedia content-based indexing

Current studies on content-based image retrieval mainly rely on bags of visual words. This model of image description allows to perform image retieval in the same way as text retrieval: documents are described as vectors of (visual) word frequencies, and documents are match by computing a distance or similarity measure between the vectors. But instead of raw frequencies, documents can also be described as vectors of word weights, each weight corresponding to the importance of the word in the document. Although the problem of determining automatically such weights, and therefore which words describe well documents, has been widely studied in the case of text retrieval, there is very little litterature applying this idea to the case of image retrieval. In this report, we explore how the use of standard weighting schemes and distance from text retrieval can help to improve the performance of image retrieval systems. We show that there is no distance or weighting scheme that can improve performance on any dataset, but choosing weights or a distance consistent with some properties of a given dataset can improve the performance up to 10%. However, we also show that in the case of very varied and general datasets, the performance gain is not significant.

Domaines: Recherche d'information [cs.IR]

Pierre Tirilly, Vincent Claveau, Patrick Gros. A review of weighting schemes for bag of visual words image retrieval. [Research Report] PI 1927, 2009, pp.47. ⟨inria-00380706⟩