EXIST 2022: Sexism detection

Binary classification problem, consisting in determmine whether a text or message is sexist or not. It includes any type of sexist expression or related phenomena, like descriptive or reported assertions where the sexist message is a report or a description of a sexist event. In particular, we consider two labels:

  • Sexist: the tweet or gab expresses sexist behaviours or discourses.
  • Non-Sexist: the tweet or gab does not express any sexist behaviour or discourse.
Publication
Francisco Rodríguez-Sánchez, Jorge Carrillo-de-Albornoz, Laura Plaza, Adrián Mendieta-Aragón, Guillermo Marco-Remón, Maryna Makeienko, María Plaza, Julio Gonzalo, Damiano Spina, Paolo Rosso (2022) Overview of EXIST 2022: sEXism Identification in Social neTworks. Procesamiento del Lenguaje Natural, Revista nº 69, septiembre de 2022, pp. 229-240.
Language
English
NLP topic
Abstract task
Dataset
Year
2022
Ranking metric
Accuracy

Task results

System Precision Recall F1 Sort ascending CEM Accuracy MacroPrecision MacroRecall MacroF1 RMSE MicroPrecision MicroRecall MicroF1 MAE MAP UAS LAS MLAS BLEX Pearson correlation Spearman correlation MeasureC BERTScore EMR Exact Match F0.5 Hierarchical F ICM MeasureC Propensity F Reliability Sensitivity Sentiment Graph F1 WAC b2 erde30 sent weighted f1
Roberta large 0.8187 0.8187 0.8187 0.8187 0.82
Xlm roberta large 0.7953 0.7953 0.7953 0.7953 0.80
Roberta base 0.7875 0.7875 0.7875 0.7875 0.79
Distilbert base uncased 0.7739 0.7739 0.7739 0.7739 0.77
Xlm roberta base 0.7661 0.7661 0.7661 0.7661 0.77
Bert base cased 0.7641 0.7641 0.7641 0.7641 0.76
Bert base multilingual cased 0.7563 0.7563 0.7563 0.7563 0.76
Ixa ehu ixambert base cased 0.7563 0.7563 0.7563 0.7563 0.76
Distilbert base multilingual cased 0.7388 0.7388 0.7388 0.7388 0.74

If you have published a result better than those on the list, send a message to odesia-comunicacion@lsi.uned.es indicating the result and the DOI of the article, along with a copy of it if it is not published openly.