The task aims at finding the best techniques to identify and categorize propagandistic tweets from governmental and diplomatic sources on a dataset of tweets in Englsih, posted by authorities of China, Russia, United States and the European Union. The task seeks to classify the tweet into four clusters of propaganda techniques: appeal to commonality, discrediting the opponent, loaded language, appeal to authority.
Publication
Pablo Moral, Guillermo Marco, Julio Gonzalo, Jorge Carrillo-de-Albornoz, Iván Gonzalo-Verdugo (2023) Overview of DIPROMATS 2023: automatic detection and characterization of propaganda techniques in messages from diplomats and authorities of world powers. Procesamiento del Lenguaje Natural, Revista nº 71, septiembre de 2023, pp. 397-407.
Language
English
NLP topic
Abstract task
Dataset
Year
2023
Publication link
Ranking metric
ICM
Task results
| System | ICM Norm Sort ascending |
|---|---|
| Mistral-7B-v03 | 0.5539 |
| Llama-3.1-8B | 0.5414 |
| Qwen2.5-7B | 0.5359 |
| Roberta large | 0.5204 |
| Xlm roberta large | 0.4867 |
| Roberta base | 0.4811 |
| Bert base cased | 0.4468 |
| Ixa ehu ixambert base cased | 0.4430 |
| Xlm roberta base | 0.4329 |
| Bert base multilingual cased | 0.4266 |
