Overview of the CLEF 2024 LongEval Lab on Longitudinal Evaluation of Model Performance

会议论文
发表日期：

2024年

查看全文

Rabab Alkhalifa,
Hsuvas Borkakoty,
Romain Deveaud,
Alaa El-Ebshihy,
Luis Espinosa-Anke,
Tobias Fink,
Petra Galuščáková,
Gabriela Gonzalez-Saez,
Lorraine Goeuriot,
David Iommi,
Maria Liakata,
Harish Tayyar Madabushi,
Pablo Medina-Alias,
Philippe Mulhem,
Florina Piroi,
Martin Popel,
Arkaitz Zubiaga

摘要

We describe the second edition of the LongEval CLEF 2024 shared task. This lab evaluates the temporal persistence of Information Retrieval (IR) systems and Text Classifiers. Task 1 requires IR systems to run on corpora acquired at several timestamps, and evaluates the drop in system quality (NDCG) along these timestamps. Task 2 tackles binary sentiment classification at different points in time, and evaluates the performance drop for different temporal gaps. Overall, 37 teams registered for Task 1 and 25 for Task 2. Ultimately, 14 and 4 teams participated in Task 1 and Task 2, respectively.