<p>Image memorability refers to the phenomenon where certain images are more likely to be remembered than others. It is a quantifiable and intrinsic image attribute, defined as the likelihood of an image being remembered upon a single exposure. Despite advances in understanding human visual perception and memory, it is unclear what features contribute to an image’s memorability. To address this question, we propose a deep learning-based computational modeling approach. We employ an autoencoder-based approach built on convolutional neural networks (CNNs). We examine the relationship between autoencoder reconstruction error and memorability, analyze the distinctiveness of latent space representations, and train a multi-layer perceptron (MLP) on the encoder’s latent representations for memorability prediction. Additionally, we perform interpretability analysis using Integrated Gradients (IG) to visualize the key visual characteristics that are most influential in our memorability prediction model. Our results demonstrate a significant correlation between the images’ memorability score and the autoencoder’s reconstruction error, as well as the robust predictive performance of its latent representations. Latent space distinctiveness correlated significantly with memorability. Additionally, certain visual characteristics were identified as features contributing to image memorability prediction in our model. These findings suggest that autoencoder-based representations are associated with image memorability, providing new insights into the computational modeling of memorability.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Modeling visual memorability assessment with autoencoders reveals characteristics of memorable images

  • Elham Bagheri,
  • Yalda Mohsenzadeh

摘要

Image memorability refers to the phenomenon where certain images are more likely to be remembered than others. It is a quantifiable and intrinsic image attribute, defined as the likelihood of an image being remembered upon a single exposure. Despite advances in understanding human visual perception and memory, it is unclear what features contribute to an image’s memorability. To address this question, we propose a deep learning-based computational modeling approach. We employ an autoencoder-based approach built on convolutional neural networks (CNNs). We examine the relationship between autoencoder reconstruction error and memorability, analyze the distinctiveness of latent space representations, and train a multi-layer perceptron (MLP) on the encoder’s latent representations for memorability prediction. Additionally, we perform interpretability analysis using Integrated Gradients (IG) to visualize the key visual characteristics that are most influential in our memorability prediction model. Our results demonstrate a significant correlation between the images’ memorability score and the autoencoder’s reconstruction error, as well as the robust predictive performance of its latent representations. Latent space distinctiveness correlated significantly with memorability. Additionally, certain visual characteristics were identified as features contributing to image memorability prediction in our model. These findings suggest that autoencoder-based representations are associated with image memorability, providing new insights into the computational modeling of memorability.