Generating Sensory Vocabulary Cues for Describing Scents in Olfactory-Enhanced Second Language Vocabulary Learning
摘要
This paper presents an approach to second language vocabulary learning that integrates olfactory stimuli with abstract visual imagery. Drawing on the Proust phenomenon, which highlights the strong link between smell and memory, as well as multisensory learning principles, we aim to improve vocabulary retention through the synergy of smell and imagery. Our system pairs scents with AI-generated abstract images to create multisensory learning cues, supported by theoretical frameworks such as bilingual dual coding theory and the encoding specificity principle. Preliminary experiments with 10 participants indicate that the effectiveness of our method depends on the congruence between the odor and the visual image. To better understand and guide this congruence, we developed a visualization technique that maps olfactory perceptions using word embeddings and dimensionality reduction. This approach reveals distinct clusters of both human- and AI-perceived odor descriptions, allowing us to pinpoint where these perceptions align or diverge. By systematically identifying more congruent odor-image pairings, the visualization provides insights for creating richer, more effective sensory-based vocabulary learning experiences.