Scholarlink

<p>Audio-text cross-modal retrieval faces challenges including heterogeneous modality discrepancies, semantic misalignment, and computational inefficiency. We propose CBP-CLAP (Compact Bilinear Pooling Contrastive Language-Audio Pretraining), which extends Contrastive Language-Audio Pretraining (CLAP) with a compact bilinear pooling (CBP) module. This module consists of four key components: (1) Count Sketch for dimensionality reduction, (2) compact bilinear pooling for feature fusion, (3) a recalibration network for feature optimization, and (4) L2 normalization for stability. Experiments on AudioCaps and Clotho demonstrate state-of-the-art performance. On AudioCaps, CBP-CLAP achieves R@1 of 42.8% (text-to-audio) and 60.1% (audio-to-text), surpassing Audio-Text+M2D2 by +0.9pp and +0.8pp respectively. For R@5 and R@10, it attains 77.3%/89.9% (text-to-audio) and 84.3%/93.8% (audio-to-text), with R@10 improvements of +1.3pp and +1.0pp. On Clotho, it achieves R@1 of 21.4%/27.8%, R@5 of 45.9%/56.9%, and R@10 of 60.7%/68.3%, outperforming M2D2 by +1.3pp/+1.5pp (R@1) and +1.2pp/+3.3pp (R@10). These results validate substantial performance gains over existing methods in audio-text cross-modal retrieval.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CBP-CLAP: Audio-text retrieval based on compact bilinear pooling for contrastive language-audio pre-training

  • 期刊论文

  • 发表日期：

    2026年4月24日

查看全文
  • Xinmin Cheng,
  • Yucheng Yuan,
  • Ruiqin Wang

摘要

Audio-text cross-modal retrieval faces challenges including heterogeneous modality discrepancies, semantic misalignment, and computational inefficiency. We propose CBP-CLAP (Compact Bilinear Pooling Contrastive Language-Audio Pretraining), which extends Contrastive Language-Audio Pretraining (CLAP) with a compact bilinear pooling (CBP) module. This module consists of four key components: (1) Count Sketch for dimensionality reduction, (2) compact bilinear pooling for feature fusion, (3) a recalibration network for feature optimization, and (4) L2 normalization for stability. Experiments on AudioCaps and Clotho demonstrate state-of-the-art performance. On AudioCaps, CBP-CLAP achieves R@1 of 42.8% (text-to-audio) and 60.1% (audio-to-text), surpassing Audio-Text+M2D2 by +0.9pp and +0.8pp respectively. For R@5 and R@10, it attains 77.3%/89.9% (text-to-audio) and 84.3%/93.8% (audio-to-text), with R@10 improvements of +1.3pp and +1.0pp. On Clotho, it achieves R@1 of 21.4%/27.8%, R@5 of 45.9%/56.9%, and R@10 of 60.7%/68.3%, outperforming M2D2 by +1.3pp/+1.5pp (R@1) and +1.2pp/+3.3pp (R@10). These results validate substantial performance gains over existing methods in audio-text cross-modal retrieval.