Sound signatures for images and geometric shapes
摘要
In this paper, we address cross-modal perception principles of sound symbolism. We propose sound signatures—short sound descriptions of images and shapes in a form of pseudowords. We hypothesize that the sound signatures can be automatically derived from geometric motives of images and shapes. The core part of this automatic generation is in computation of what we call “dominant angularity” of the images and shapes. We then propose: (1) a geometry processing pipeline that calculates the dominant angularity of geometric shapes; (2) a deep learning framework that predicts the dominant angularity matching geometric shapes for sound signatures given in text and audio formats; and (3) a deep learning model that effectively generates a set of sound signatures based on dominant angularity inputs. We successfully reproduced and used as a test case the century-long user studies on sound symbolism, which demonstrated the robust cross-modal performance of our methods. Our results highlight the importance of sound envelopes and angular features in these synesthetic associations, and enable perception-aligned naming systems using sound signatures.