A general framework for the evaluation of symbol recognition methods

Ernest Valveny (1) , Philippe Dosch (2) , Adam Winstanley , Zhou Yu , Yang Su , Yan Luo , Wenyin Liu , Dave Elliman , Mathieu Delalandre , Eric Trupin , Sébastien Adam , Jean-Marc Ogier

1 CVC - Computer Vision Center (Centre de visio per computador)
2 QGAR - Querying Graphics through Analysis and Recognition

Performance evaluation is receiving increasing interest in graphics recognition. In this paper, we discuss some questions regarding the definition of a general framework for evaluation of symbol recognition methods. The discussion is centered on three key elements in performance evaluation: test data, evaluation metrics and protocols of evaluation. As a result of this discussion we state some general principles to be taken into account for the definition of such a framework. Finally, we describe the application of this framework to the organization of the first contest on symbol recognition in GREC'03, along with the results obtained by the participants.

Mots clés: Performance evaluation; Symbol recognition

Domaines: Traitement des images [eess.IV]; Traitement du texte et du document

Ernest Valveny, Philippe Dosch, Adam Winstanley, Zhou Yu, Yang Su, et al.. A general framework for the evaluation of symbol recognition methods. International Journal on Document Analysis and Recognition, 2007, 9 (1), pp.59-74. ⟨10.1007/s10032-006-0033-x⟩. ⟨inria-00176899⟩