<p>In this paper, we address cross-modal perception principles of sound symbolism. We propose sound signatures—short sound descriptions of images and shapes in a form of pseudowords. We hypothesize that the sound signatures can be automatically derived from geometric motives of images and shapes. The core part of this automatic generation is in computation of what we call “dominant angularity” of the images and shapes. We then propose: (1) a geometry processing pipeline that calculates the dominant angularity of geometric shapes; (2) a deep learning framework that predicts the dominant angularity matching geometric shapes for sound signatures given in text and audio formats; and (3) a deep learning model that effectively generates a set of sound signatures based on dominant angularity inputs. We successfully reproduced and used as a test case the century-long user studies on sound symbolism, which demonstrated the robust cross-modal performance of our methods. Our results highlight the importance of sound envelopes and angular features in these synesthetic associations, and enable perception-aligned naming systems using sound signatures.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Sound signatures for images and geometric shapes

  • Hanqin Wang,
  • Alexei Sourin

摘要

In this paper, we address cross-modal perception principles of sound symbolism. We propose sound signatures—short sound descriptions of images and shapes in a form of pseudowords. We hypothesize that the sound signatures can be automatically derived from geometric motives of images and shapes. The core part of this automatic generation is in computation of what we call “dominant angularity” of the images and shapes. We then propose: (1) a geometry processing pipeline that calculates the dominant angularity of geometric shapes; (2) a deep learning framework that predicts the dominant angularity matching geometric shapes for sound signatures given in text and audio formats; and (3) a deep learning model that effectively generates a set of sound signatures based on dominant angularity inputs. We successfully reproduced and used as a test case the century-long user studies on sound symbolism, which demonstrated the robust cross-modal performance of our methods. Our results highlight the importance of sound envelopes and angular features in these synesthetic associations, and enable perception-aligned naming systems using sound signatures.