Investigating the document structure as a source of evidence for multimedia fragment retrieval

Mouna Torjmen-Khemakhem (1) , Karen Pinel-Sauvagnat (2) , Mohand Boughanem (2, 3)

1 ReDCAD - Research Unit on Development and Control of Distributed Applications
2 IRIT-SIG - Systèmes d’Informations Généralisées
3 UT3 - Université Toulouse III - Paul Sabatier

Multimedia objects can be retrieved using their context that can be for instance the text surrounding them in documents. This text may be either near or far from the searched objects. Our goal in this paper is to study the impact, in term of effectiveness, of text position relatively to searched objects. The multimedia objects we consider are described in structured documents such as XML ones. The document structure is therefore exploited to provide this text position in documents. Although structural information has been shown to be an effective source of evidence in textual information retrieval, only a few works investigated its interest in multimedia retrieval. More precisely, the task we are interested in this paper is to retrieve multimedia fragments (i.e. XML elements having at least one multimedia object). Our general approach is built on two steps: we first retrieve XML elements containing multimedia objects, and we then explore the surrounding information to retrieve relevant multimedia fragments. In both cases, we study the impact of the surrounding information using the documents structure.Our work is carried out on images, but it can be extended to any other media, since the physical content of multimedia objects is not used. We conducted several experiments in the context of the Multimedia track of the INEX evaluation campaign. Results showed that structural evidences are of high interest to tune the importance of textual context for multimedia retrieval. Moreover, the proposed approach outperforms state of the art approaches.

Mots clés: XML document; Context-based image retrieval; Multimedia fragment; Hierarchical structure

Domaines: Théorie de l'information [cs.IT]; Recherche d'information [cs.IR]

Mouna Torjmen-Khemakhem, Karen Pinel-Sauvagnat, Mohand Boughanem. Investigating the document structure as a source of evidence for multimedia fragment retrieval. Information Processing and Management, 2013, vol. 49 (n° 6), pp. 1281-1300. ⟨10.1016/j.ipm.2013.06.001⟩. ⟨hal-01131253⟩