Scholarlink

Large language models (LLMs) have become foundational to numerous natural language processing tasks; however, decoding coherent and contextually relevant text remains a complex challenge. In open-ended generation, maximizing probability is often not the appropriate objective, as with sampling methods, the continuation tends to be incoherent and repetitive in various degrees. We propose Merge Decoding, merging information in the shallow layer, such as sequential information, with the final task-specific layer, thereby generating coherent and rich text. MD works across three scales of the LLaMA family(7B, 13B, 30B), achieving higher quality text in open-ended text generation (WikiText, WikiNews, BookCorpus) and enhancing reasoning capabilities in downstream tasks (Gsm8k, StrategyQA) https://github.com/YcChou/MergeDecoding .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

The Benefits in Shallow: Merge Decoding Across Large Language Model Layers

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yuechi Zhou,
  • Chuyue Zhou,
  • Wenjing Xie,
  • Xinrui Wang,
  • Jiuchang Chen,
  • Zhenghua Ni,
  • Juntao Li

摘要

Large language models (LLMs) have become foundational to numerous natural language processing tasks; however, decoding coherent and contextually relevant text remains a complex challenge. In open-ended generation, maximizing probability is often not the appropriate objective, as with sampling methods, the continuation tends to be incoherent and repetitive in various degrees. We propose Merge Decoding, merging information in the shallow layer, such as sequential information, with the final task-specific layer, thereby generating coherent and rich text. MD works across three scales of the LLaMA family(7B, 13B, 30B), achieving higher quality text in open-ended text generation (WikiText, WikiNews, BookCorpus) and enhancing reasoning capabilities in downstream tasks (Gsm8k, StrategyQA) https://github.com/YcChou/MergeDecoding .