DIANN 2023: Disability detection

The task consists on detecting disability mentions in abstracts of biomedical articles in Spanish. The task follows the guidelines established in the IberLEF 2018 competition "Disability annotation on documents from the biomedical domain (DIANN)".

Language
Spanish
NLP topic
(named) entity recognition
Abstract task
Sequence labeling
Dataset
DIANN-2023-ES
Year
2023
Ranking metric
F1

Task results

System Precision Recall F1 CEM Accuracy MacroPrecision MacroRecall MacroF1 RMSE MicroPrecision MicroRecall MicroF1 MAE MAP UAS LAS MLAS BLEX Pearson correlation Spearman correlation MeasureC BERTScore EMR Exact Match F0.5 Hierarchical F ICM MeasureC Propensity F Reliability Sensitivity Sentiment Graph F1 WAC b2 erde30 sent weighted f1
Xlm roberta large 0.7855 0.7855 0.7855 0.7855 0.79
Xlm roberta base 0.7819 0.7819 0.7819 0.7819 0.78
Bert base multilingual cased 0.7592 0.7592 0.7592 0.7592 0.76
Ixa ehu ixambert base cased 0.7580 0.7580 0.6868 0.7580 0.76
Dccuchile bert base spanish wwm cased 0.7478 0.7478 0.7478 0.7478 0.75
PlanTL GOB ES roberta base bne 0.7169 0.7169 0.7169 0.7169 0.72
Hermes-3-Llama-3.1-8B 0.7056 0.7056 0.7056 0.7056 0.71
Hermes-3-Llama-3.1-8B_2 0.7042 0.7042 0.7042 0.7042 0.70
Bertin roberta base spanish 0.6877 0.6877 0.6877 0.6877 0.69
Distilbert base multilingual cased 0.6868 0.6868 0.6868 0.6868 0.69

If you have published a result better than those on the list, send a message to odesia-comunicacion@lsi.uned.es indicating the result and the DOI of the article, along with a copy of it if it is not published openly.