This is an extractive text comprehension task formulated in terms of question-answering. The task consists of answering questions about a text in such a way that the answer is a fragment extracted directly from the text. The texts are academic news from Cambridge University from several scientific domains. In all cases, the answers are fragments of the text and questions that cannot be answered from the text are not included.
Language
English
NLP topic
Abstract task
Dataset
Year
2024
Ranking metric
F1
Task results
| System | F1 |
|---|---|
| distilbert-base-multilingual-cased | 0.1990 |
| distilbert-base-uncased | 0.2670 |
| bert-base-cased | 0.3000 |
| ixambert-base-cased | 0.3220 |
| xlm-roberta-base | 0.3250 |
| bert-base-multilingual-cased | 0.2950 |
| roberta-base | 0.3750 |
| xlm-roberta-large | 0.4160 |
| roberta-large | 0.4630 |
