Scholarlink

<p>Sonification transforms visual data into sound. This study explores whether sonified images retain enough visual information for computational analysis. We ask: <b>Do sonified images preserve image information?</b> We propose a novel method that reverses the standard Griffin-Lim Algorithm (GLA). A colour image is treated as a target spectrogram, and an audio signal is iteratively reconstructed to match it. The resulting sound is converted back into a spectrogram and evaluated using pretrained convolution neural networks (CNNs) to measure how well the original image’s structure is preserved. Tests across four datasets (2-15 classes) and five CNN models show that sonified spectrograms retain significant class-discriminative information. Performance was highest for simpler, visually distinct datasets (e.g., ResNet152 achieved 83.32% accuracy for 2 classes, 90.62% for 4 classes) but declined for complex, fine-grained tasks (e.g., 48.84% for 15 classes) due to reconstruction artifacts and loss of spatial detail. These results confirm that low-level sonification can algorithmically encode meaningful visual structure, enabling downstream classification without prior semantic knowledge. This framework provides a foundation for cross-modal learning and a tool for quantitatively evaluating sonification fidelity.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

When Images Become Sound: Preserving Visual Semantics with I-GLA

  • 期刊论文

  • 发表日期：

    2026年1月28日

查看全文
  • Sreejib Pal,
  • Anirban Bhowmick

摘要

Sonification transforms visual data into sound. This study explores whether sonified images retain enough visual information for computational analysis. We ask: Do sonified images preserve image information? We propose a novel method that reverses the standard Griffin-Lim Algorithm (GLA). A colour image is treated as a target spectrogram, and an audio signal is iteratively reconstructed to match it. The resulting sound is converted back into a spectrogram and evaluated using pretrained convolution neural networks (CNNs) to measure how well the original image’s structure is preserved. Tests across four datasets (2-15 classes) and five CNN models show that sonified spectrograms retain significant class-discriminative information. Performance was highest for simpler, visually distinct datasets (e.g., ResNet152 achieved 83.32% accuracy for 2 classes, 90.62% for 4 classes) but declined for complex, fine-grained tasks (e.g., 48.84% for 15 classes) due to reconstruction artifacts and loss of spatial detail. These results confirm that low-level sonification can algorithmically encode meaningful visual structure, enabling downstream classification without prior semantic knowledge. This framework provides a foundation for cross-modal learning and a tool for quantitatively evaluating sonification fidelity.