In this chapter, we explore the latest advancements in detecting decontextualised and miscaptioned images, i.e., images whose captions or accompanying text represent their origin, context, or meaning in a false or misleading way. We examine key aspects of the field, including the development of annotated, weakly annotated, and synthetically generated datasets; advances in multimodal feature extraction, modality fusion, and detection models; approaches that incorporate external information as evidence; and the emerging role of large vision–language models in this task. Furthermore, we present our contributions within the context of the vera.ai research project, which include the creation of synthetic training datasets, real-world evaluation benchmarks, and methodologies aimed at improving model generalisation, mitigating unimodal bias, and strengthening the reliability of evidence-based verification.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Detection of Image Misrepresentation

  • Stefanos-Iordanis Papadopoulos,
  • Christos Koutlis,
  • Symeon Papadopoulos,
  • Panagiotis C. Petrantonakis

摘要

In this chapter, we explore the latest advancements in detecting decontextualised and miscaptioned images, i.e., images whose captions or accompanying text represent their origin, context, or meaning in a false or misleading way. We examine key aspects of the field, including the development of annotated, weakly annotated, and synthetically generated datasets; advances in multimodal feature extraction, modality fusion, and detection models; approaches that incorporate external information as evidence; and the emerging role of large vision–language models in this task. Furthermore, we present our contributions within the context of the vera.ai research project, which include the creation of synthetic training datasets, real-world evaluation benchmarks, and methodologies aimed at improving model generalisation, mitigating unimodal bias, and strengthening the reliability of evidence-based verification.