Few-shot Multimodal Aspect-Level Sentiment Classification (MALSC) has garnered significant attention due to the proliferation of multimodal information on social media. In-Context Learning (ICL) with Large Vision-Language Models (LVLMs) offers a promising paradigm for MALSC. However, standard ICL is severely hampered by the high token cost of images, which restricts the number of contextual demonstrations. Circumventing this by replacing images with text captions often leads to critical information loss and error propagation. To address these limitations, we propose a novel framework based on Implicit In-Context Learning (I2CL). Our method compresses multiple image-text demonstrations into a single, compact “context vector” that encapsulates the essence of the task. During inference, this vector is directly injected into the LVLM’s activation space, guiding its reasoning without consuming any input tokens. This approach not only enables the model to leverage rich, raw visual information but also fundamentally resolves the token bottleneck and caption-induced errors inherent in previous methods. Extensive experiments on the Twitter-2015 and Twitter-2017 benchmarks demonstrate that our I2CL framework significantly enhances both performance and stability for few-shot MALSC, outperforming state-of-the-art models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improving Few-Shot Multi-modal Aspect-Level Sentiment Classification with Implicit In-context Learning

  • Aoxiang Zhou,
  • Hao Wu,
  • Kaichen Peng,
  • Peng Liu,
  • Xianxian Li

摘要

Few-shot Multimodal Aspect-Level Sentiment Classification (MALSC) has garnered significant attention due to the proliferation of multimodal information on social media. In-Context Learning (ICL) with Large Vision-Language Models (LVLMs) offers a promising paradigm for MALSC. However, standard ICL is severely hampered by the high token cost of images, which restricts the number of contextual demonstrations. Circumventing this by replacing images with text captions often leads to critical information loss and error propagation. To address these limitations, we propose a novel framework based on Implicit In-Context Learning (I2CL). Our method compresses multiple image-text demonstrations into a single, compact “context vector” that encapsulates the essence of the task. During inference, this vector is directly injected into the LVLM’s activation space, guiding its reasoning without consuming any input tokens. This approach not only enables the model to leverage rich, raw visual information but also fundamentally resolves the token bottleneck and caption-induced errors inherent in previous methods. Extensive experiments on the Twitter-2015 and Twitter-2017 benchmarks demonstrate that our I2CL framework significantly enhances both performance and stability for few-shot MALSC, outperforming state-of-the-art models.