Scholarlink

<p>We present Lumina-mGPT, a family of multimodal autoregressive models capable of various vision and language tasks, particularly excelling in generating flexible photorealistic images from text descriptions. By initializing from <Emphasis Type="BoldItalic">m</Emphasis><i>ultimodal</i> <Emphasis Type="BoldItalic">G</Emphasis><i>enerative</i> <Emphasis Type="BoldItalic">P</Emphasis><i>re</i> <Emphasis Type="BoldItalic">T</Emphasis><i>raining</i> (mGPT), we demonstrate that decoder-only Autoregressive (AR) model can achieve image generation performance comparable to modern diffusion model with high efficiency through <i>Flexible Progressive Supervised Finetuning</i> (FP-SFT). Equipped with our proposed <Emphasis Type="BoldItalic">Un</Emphasis><i>ambiguous</i> <Emphasis Type="BoldItalic">i</Emphasis><i>mage</i> <Emphasis Type="BoldItalic">Rep</Emphasis><i>resentation</i> (Uni-Rep), Lumina-mGPT can flexibly generate high-quality images of varying aspect ratios. Building on the strong image generation capabilities, we further explore <i>Ominipotent Supervised Finetuning</i> (Omni-SFT), an initial attempt to elevate Lumina-mGPT into a unified multi-modal generalist. The resulting model demonstrates versatile multimodal capabilities, including visual generation tasks like text-to-image/multiview generation and controllable generation, visual recognition tasks like segmentation and depth estimation, and vision-language tasks like multi-turn visual question answering, showing the rosy potential of the technical direction. We release all code and checkpoints at <a href="https://github.com/Alpha-VLLM/Lumina-mGPT">https://github.com/Alpha-VLLM/Lumina-mGPT</a>, hoping to facilitate the progress towards AR-based multi-modal generalist.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Lumina-mGPT: Flexible Photorealistic Autoregressive Text-to-Image Generation

  • 期刊论文

  • 发表日期：

    2026年3月5日

查看全文
  • Dongyang Liu,
  • Yi Xin,
  • Shitian Zhao,
  • Le Zhuo,
  • Weifeng Lin,
  • Xinyue Li,
  • Qi Qin,
  • Guangtao Zhai,
  • Xiaohong Liu,
  • Hongsheng Li,
  • Yu Qiao,
  • Peng Gao

摘要

We present Lumina-mGPT, a family of multimodal autoregressive models capable of various vision and language tasks, particularly excelling in generating flexible photorealistic images from text descriptions. By initializing from multimodal Generative Pre Training (mGPT), we demonstrate that decoder-only Autoregressive (AR) model can achieve image generation performance comparable to modern diffusion model with high efficiency through Flexible Progressive Supervised Finetuning (FP-SFT). Equipped with our proposed Unambiguous image Representation (Uni-Rep), Lumina-mGPT can flexibly generate high-quality images of varying aspect ratios. Building on the strong image generation capabilities, we further explore Ominipotent Supervised Finetuning (Omni-SFT), an initial attempt to elevate Lumina-mGPT into a unified multi-modal generalist. The resulting model demonstrates versatile multimodal capabilities, including visual generation tasks like text-to-image/multiview generation and controllable generation, visual recognition tasks like segmentation and depth estimation, and vision-language tasks like multi-turn visual question answering, showing the rosy potential of the technical direction. We release all code and checkpoints at https://github.com/Alpha-VLLM/Lumina-mGPT, hoping to facilitate the progress towards AR-based multi-modal generalist.