This is an extractive text comprehension task formulated in terms of question-answering. The task consists of answering questions about a text in such a way that the answer is a fragment extracted directly from the text. The texts are academic news from the CSIC (Centro Superior de Investigaciones Científicas) from several scientific domains. In all cases, the answers are fragments of the text and questions that cannot be answered from the text are not included.
Language
Spanish
NLP topic
Abstract task
Dataset
Year
2024
Ranking metric
F1
Task results
| System | F1 Sort ascending |
|---|---|
| Qwen2.5-14B-Instruct | 0.7265 |
| Qwen2.5-14B-Instruct_mix | 0.7265 |
| Hermes-3-Llama-3.1-8B | 0.6791 |
| Hermes-3-Llama-3.1-8B_mix_4 | 0.6791 |
| Hermes-3-Llama-3.1-8B_2 | 0.6791 |
| Hermes-3-Llama-3.1-8B_mix_5 | 0.6791 |
| Hermes-3-Llama-3.1-8B_mix_6 | 0.6791 |
| Llama_3.1-8B-Instruct 0 shot no BIO v2 | 0.6565 |
| Llama_3.1-8B-Instruct 0 shot no BIO v4 | 0.6554 |
| Llama_3.1-8B-Instruct 0 shot no BIO v3 | 0.6554 |
