Given a text, a question on that text, and a set of candidate answers, a system must select the correct answer from a set of 2-7 candidate answers. Additionally, each question-answer pair instance includes a short explanation as reasoning support for choosing a candidate answer.
Publication
Marco Antonio Sobrevilla Cabezudo, Diego Diestra, Rodrigo López, Erasmo Gómez, Arturo Oncevay, Fernando Alva-Manchego (2022) Overview of ReCoRES at IberLEF 2022: Reading Comprehension and Reasoning Explanation for Spanish. Procesamiento del Lenguaje Natural, Revista nº 69, septiembre de 2022, pp. 281-287.
Language
Spanish
NLP topic
Abstract task
Year
2022
Publication link
Ranking metric
Accuracy
Task results
| System | Precision | Recall | F1 | CEM | Accuracy Sort ascending | MacroPrecision | MacroRecall | MacroF1 | RMSE | MicroPrecision | MicroRecall | MicroF1 | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MRCPUCP | 0.7591 | ||||||||||||||||||||||||||||||||||||
| SADDA | 0.7254 | ||||||||||||||||||||||||||||||||||||
| Versae & Nandezgarcia | 0.4067 |
