Third Recognising Textual Entailment Challenge

RTE 3 follows the same basic structure of the previous campaign, in order to facilitate the participation of newcomers and to allow “veterans” to assess the improvements of their systems in a comparable test exercise. Nevertheless, the following innovations are introduced to make the challenge more stimulating and, at the same time, to encourage collaboration between system developers:

• a limited number of longer texts, i.e. up to a paragraph- in order to move toward more comprehensive scenarios which incorporate the need for discourse analysis. However, the majority of examples will remain similar to those in the previous challenges, providing pairs with relatively short texts.
• an RTE Resource Pool has been created where contributors have the possibility to share the resources they use.
• an optional task, “Extending the Evaluation of Inferences from Texts”, which explores two other tasks closely related to textual entailment: differentiating unknown from false/contradicts and providing justifications for answers.