EXIST is a series of scientific events and shared tasks on sexism identification in social networks. EXIST aims to capture sexism in a broad sense, from explicit misogyny to other subtle expressions that involve implicit sexist behaviours.This task includes a soft-soft evaluation in which the probability of each label predicted by the system is compared with the probability defined from the disagreement in the gold standard annotation.
Publication
Plaza, L. et al. (2023). Overview of EXIST 2023 – Learning with Disagreement for Sexism Identification and Characterization. In: Arampatzis, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2023. Lecture Notes in Computer Science, vol 14163. Springer, Cham. https://doi.org/10.1007/978-3-031-42448-9_23
Competition
Language
English
URL Task
NLP topic
Abstract task
Dataset
Year
2023
Publication link
Ranking metric
ICM
Task results
System | F1 Sort ascending | Accuracy | MacroF1 | Pearson correlation | ICM |
---|---|---|---|---|---|
Xlm roberta large | 0.4029 | 0.4029 | 0.4029 | 0.4029 | 0.40 |
Roberta large | 0.3962 | 0.3962 | 0.3962 | 0.3962 | 0.40 |
Distilbert base uncased | 0.3844 | 0.3844 | 0.3844 | 0.3844 | 0.38 |
Roberta base | 0.3779 | 0.3779 | 0.3779 | 0.3779 | 0.38 |
Xlm roberta base | 0.3735 | 0.3735 | 0.3735 | 0.3735 | 0.37 |
Bert base cased | 0.3701 | 0.3701 | 0.3701 | 0.3701 | 0.37 |
Ixa ehu ixambert base cased | 0.3622 | 0.3622 | 0.3622 | 0.3622 | 0.36 |
Bert base multilingual cased | 0.3618 | 0.3618 | 0.3618 | 0.3618 | 0.36 |
Distilbert base multilingual cased | 0.3576 | 0.3576 | 0.3576 | 0.3576 | 0.36 |