Syntactic testsuites and Textual Entailment Recognition

Paul Bedaride (1) , Claire Gardent (1)

1 TALARIS - Natural Language Processing: representation, inference and semantics

We focus on textual entailments mediated by syntax and propose a new methodology to evaluate textual entailment recognition systems on such data. The main idea is to generate a syntactically annotated corpus of pairs of (non-)entailments and to use error mining to identify the most likely sources of errors. To illustrate the approach, we apply this methodology to the Afazio RTE system and show how it permits identifying the most likely sources of errors made by this system on a testsuite of 10 000 (non) entailment pairs.

Mots clés: testsuites recognising textual entailment error mining

Domaines: Informatique et langage [cs.CL]

Paul Bedaride, Claire Gardent. Syntactic testsuites and Textual Entailment Recognition. The seventh International Conference on Language Resources and Evaluation - LREC 2010, May 2010, Valletta, Malta. ⟨inria-00536020⟩