Scholarlink

Humans can easily solve tasks involving different types of information, such as text, images, or sounds, using just a few examples or simple instructions. However, current multimodal systems have been struggling to mimic this ability. In this chapter, we present Emu2, a multimodal modelMultimodal model with 37 billion parameters, designed to improve this challenge. Trained on large-scale multimodal data, Emu2 shows strong abilities in learning from context and can handle tasks that require reasoning on the spot, such as visual promptingVisual prompting and object-based generation. Emu2 sets new performance records on several multimodal understanding tasks, even in situations where only a few examples are provided. Additionally, when fine-tuned with specific instructions, it achieves state-of-the-art results on tasks like question answering and open-ended content generation. These advancements demonstrate that Emu2 is a versatile model, capable of serving as a foundation for a wide variety of multimodal tasks. Its availability to the public will support future research and development in this area.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Generative Multimodal Models Are In-Context Learners

  • 章节

  • 发表日期：

    2026年

查看全文
  • Yufeng Cui,
  • Quan Sun,
  • Xiaosong Zhang,
  • Fan Zhang,
  • Qiying Yu,
  • Zhengxiong Luo,
  • Yueze Wang,
  • Yongming Rao,
  • Jingjing Liu,
  • Tiejun Huang,
  • Xinlong Wang

摘要

Humans can easily solve tasks involving different types of information, such as text, images, or sounds, using just a few examples or simple instructions. However, current multimodal systems have been struggling to mimic this ability. In this chapter, we present Emu2, a multimodal modelMultimodal model with 37 billion parameters, designed to improve this challenge. Trained on large-scale multimodal data, Emu2 shows strong abilities in learning from context and can handle tasks that require reasoning on the spot, such as visual promptingVisual prompting and object-based generation. Emu2 sets new performance records on several multimodal understanding tasks, even in situations where only a few examples are provided. Additionally, when fine-tuned with specific instructions, it achieves state-of-the-art results on tasks like question answering and open-ended content generation. These advancements demonstrate that Emu2 is a versatile model, capable of serving as a foundation for a wide variety of multimodal tasks. Its availability to the public will support future research and development in this area.