Overview of the CLEF-2024 CheckThat! Lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness

Mostra el registre complet Registre parcial de l'ítem

  • dc.contributor.author Barrón Cedeño, Alberto
  • dc.contributor.author Alam, Firoj
  • dc.contributor.author Struß, Julia Maria
  • dc.contributor.author Nakov, Preslav
  • dc.contributor.author Chakraborty, Tanmoy
  • dc.contributor.author Elsayed, Tamer
  • dc.contributor.author Przybyła, Piotr
  • dc.contributor.author Caselli, Tommaso
  • dc.contributor.author San Martino, Giovanni da
  • dc.contributor.author Haouari, Fatima
  • dc.contributor.author Hasanain, Maram
  • dc.contributor.author Li, Chengkai
  • dc.contributor.author Piskorski, Jakub
  • dc.contributor.author Ruggeri, Federico
  • dc.contributor.author Song, Xingyi
  • dc.contributor.other Suwaileh, Reem
  • dc.date.accessioned 2024-10-15T06:15:11Z
  • dc.date.embargoEnd info:eu-repo/date/embargoEnd/2025-09-09
  • dc.date.issued 2024
  • dc.description.abstract We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF). Previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, identifying previously factchecked claims, supporting evidence retrieval, and claim verification. In this edition, we introduced some new challenges, offering six tasks in fifteen languages (Arabic, Bulgarian, English, Dutch, French, Georgian, German, Greek, Italian, Polish, Portuguese, Russian, Slovene, Spanish, and code-mixed Hindi-English): Task 1 on estimation of check-worthiness (the only task that has been present in all CheckThat! editions), Task 2 on identification of subjectivity (a follow up of the CheckThat! 2023 edition), Task 3 on identification of the use of persuasion techniques (a follow up of SemEval 2023), Task 4 on detection of hero, villain, and victim from memes (a follow up of CONSTRAINT 2022), Task 5 on rumor verification using evidence from authorities (new task), and Task 6 on robustness of credibility assessment with adversarial examples (new task). These are challenging classification and retrieval problems at the document and at the span level, including multilingual and multimodal settings. This year, CheckThat! was one of the most popular labs at CLEF-2024 in terms of team registrations: 130 teams. More than onethird of them (a total of 46) actually participated.
  • dc.embargo.liftdate 2025-09-09
  • dc.format.mimetype application/pdf
  • dc.identifier.citation Barrón-Cedeño A, Alam F, Struß JM, Nakov P, Chakraborty T, Elsayed T, et al. Overview of the CLEF-2024 CheckThat! Lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness. In: Goeuriot L, Mulhem P; Quénot G; Schwab D; Di Nunzio GM; Soulier L, et al., editors. Experimental IR meets multilinguality, multimodality, and interaction. 15th International Conference of the CLEF Association 2024; 2024 September 9–12; Grenoble, France. Cham: Springer; 2024. p. 28-52
  • dc.identifier.uri http://hdl.handle.net/10230/61393
  • dc.language.iso eng
  • dc.publisher Springer
  • dc.relation.projectID info:eu-repo/grantAgreement/EC/HE/101060930
  • dc.rights © 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
  • dc.rights.accessRights info:eu-repo/semantics/embargoedAccess
  • dc.subject.keyword Fact-checking
  • dc.subject.keyword Check-worthiness
  • dc.subject.keyword Subjectivity
  • dc.subject.keyword Propaganda
  • dc.subject.keyword Rumor verification
  • dc.subject.keyword Credibility assessment
  • dc.subject.keyword Authority finding
  • dc.title Overview of the CLEF-2024 CheckThat! Lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness
  • dc.type info:eu-repo/semantics/conferenceObject
  • dc.type.version info:eu-repo/semantics/acceptedVersion