This is a multi-class classification tasks. The systems have to decide whether or not a given tweet contains sexist expressions or behaviours (i.e., it is sexist itself, describes a sexist situation or criticizes a sexist behaviour) and, if so, to categorize the message according to the type of sexism (according to the categorization proposed by experts and that takes into account the different facets of women that are undermined): (i) ideological and inequality, (ii) stereotyping and dominance, (iii) objectification, (iv) sexual violence, and (v) misogyny and non-sexual violence.
Publication
Francisco Rodríguez-Sánchez, Jorge Carrillo-de-Albornoz, Laura Plaza, Adrián Mendieta-Aragón, Guillermo Marco-Remón, Maryna Makeienko, María Plaza, Julio Gonzalo, Damiano Spina, Paolo Rosso (2022) Overview of EXIST 2022: sEXism Identification in Social neTworks. Procesamiento del Lenguaje Natural, Revista nº 69, septiembre de 2022, pp. 229-240.
Language
Spanish
URL Task
NLP topic
Abstract task
Dataset
Year
2022
Publication link
Ranking metric
Macro F1
Task results
| System | Precision | Recall | F1 Sort ascending | CEM | Accuracy | MacroPrecision | MacroRecall | MacroF1 | RMSE | MicroPrecision | MicroRecall | MicroF1 | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hermes-3-Llama-3.1-8B_2 | 0.5736 | 0.5736 | 0.5736 | 0.5736 | 0.57 | ||||||||||||||||||||||||||||||||
| Hermes-3-Llama-3.1-8B | 0.5736 | 0.5736 | 0.5736 | 0.5736 | 0.57 | ||||||||||||||||||||||||||||||||
| PlanTL GOB ES roberta large bne | 0.5668 | 0.5668 | 0.5668 | 0.5668 | 0.57 | ||||||||||||||||||||||||||||||||
| Xlm roberta large | 0.5593 | 0.5593 | 0.5593 | 0.5593 | 0.56 | ||||||||||||||||||||||||||||||||
| PlanTL GOB ES roberta base bne | 0.5554 | 0.5554 | 0.5554 | 0.5554 | 0.56 | ||||||||||||||||||||||||||||||||
| XLM-RoBERTa-large | 0.5540 | 0.5540 | 0.5540 | 0.5540 | 0.55 | ||||||||||||||||||||||||||||||||
| XLM-RoBERTa-large-2 | 0.5540 | 0.5540 | 0.5540 | 0.5540 | 0.55 | ||||||||||||||||||||||||||||||||
| XLM-RoBERTa-large-v3 | 0.5540 | 0.5540 | 0.5540 | 0.5540 | 0.55 | ||||||||||||||||||||||||||||||||
| Dccuchile bert base spanish wwm cased | 0.5370 | 0.5370 | 0.5370 | 0.5370 | 0.54 | ||||||||||||||||||||||||||||||||
| Gemma-2B-IT | 0.5262 | 0.5262 | 0.5262 | 0.5262 | 0.53 |
