Global vector-based methods for image retrieval often fail to capture the fine-grained interactions between objects in complex scenes. We present VKG-IR, a reliability-aware visual graph retrieval framework that represents each image as a set of visual triplets extracted by a pretrained scene-graph generator. VKG-IR includes a reliability module that estimates a reliability score for each triplet and uses these scores to reweight relations during structural aggregation, reducing the impact of inaccurate or redundant triplets on the image representation. The model is trained with a joint objective that combines an image-level ranking loss and a relation-level reliability supervision term. At inference, a query image is encoded once and ranked against indexed embeddings using cosine similarity. Experiments on the VG-COCO dataset under an IDF-weighted relevance labeling protocol show that VKG-IR improves Recall@K and mAP over global and graph-based baselines. Ablation studies further examine the contribution of the reliability mechanism and the linear fusion of graph and global representations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VKG-IR: Reliability-Aware Visual Graph Embedding for Fine-Grained Image Retrieval

  • Nam Nguyen,
  • Long Nguyen,
  • Thanh Le

摘要

Global vector-based methods for image retrieval often fail to capture the fine-grained interactions between objects in complex scenes. We present VKG-IR, a reliability-aware visual graph retrieval framework that represents each image as a set of visual triplets extracted by a pretrained scene-graph generator. VKG-IR includes a reliability module that estimates a reliability score for each triplet and uses these scores to reweight relations during structural aggregation, reducing the impact of inaccurate or redundant triplets on the image representation. The model is trained with a joint objective that combines an image-level ranking loss and a relation-level reliability supervision term. At inference, a query image is encoded once and ranked against indexed embeddings using cosine similarity. Experiments on the VG-COCO dataset under an IDF-weighted relevance labeling protocol show that VKG-IR improves Recall@K and mAP over global and graph-based baselines. Ablation studies further examine the contribution of the reliability mechanism and the linear fusion of graph and global representations.