On the Implications of Data Contamination for Information Retrieval Systems
摘要
Data contamination occurs when test instances have been compromised during a training stage of building a machine learning model. The consequences of this phenomenon over the quality of learning data are crucial when evaluating a learned predictor, since it could distort the assessment of the actual capabilities of the system. Its study has recently gained more traction in the research on Large Language Models, where it is common to chase performances in order to support claims about model abilities. Since the field of Information Retrieval increasingly studies and develops approaches that rely on these data-centric technologies, this position paper considers the phenomenon of data contamination in terms of its possible consequences for this field.