Content-based inference of hierarchical structural grammar for recurrent TV programs using multiple sequence alignment

Bingqing Qu (1, 2) , Félicien Vallet (2) , Jean Carrive (2) , Guillaume Gravier (1)

1 TEXMEX - Multimedia content-based indexing
2 INA - Institut National de l'Audiovisuel

Recently, unsupervised approaches were introduced to analyze the structure of TV programs, relying on the discovery of repeated elements within a program or across multiple episodes of the same program. These methods can discover key repeating elements, such as jingles and separators, however they cannot infer the entire struc- ture of a program. In this paper, we propose a hierarchical use of grammatical inference to yield a temporal grammar of a program from a collection of episodes, discovering both the vocabulary of the grammar and the temporal organization of the words from the vocab- ulary. Using a set of basic event detectors and simple filtering tech- niques to detect repeating elements of interest, a symbolic represen- tation of each episode is derived based on minimal domain knowl- edge. Grammatical inference based on multiple sequence alignment is then used in a hierarchical manner to provide a temporal grammar of the program at various levels of details. Experimental validation is performed on 3 distinct types of programs on 4 datasets. Qualitative analyses show that the grammars inferred at the different levels of the hierarchy are relevant and can be obtained from a fairly limited number of episodes.

Mots clés: TV program structuring; hierarchical structural grammar; symbolic representation; multiple sequence alignment; unsupervised and multimodal approach

Domaines: Multimédia [cs.MM]

Bingqing Qu, Félicien Vallet, Jean Carrive, Guillaume Gravier. Content-based inference of hierarchical structural grammar for recurrent TV programs using multiple sequence alignment. IEEE International Conference on Multimedia and Expo, Jul 2014, Chengdu, China. ⟨hal-01026335⟩