Scholarlink

Training and fine-tuning language models is becoming increasingly expensive. “Model soups” offer a promising solution by combining parameters from separately trained models to create a new one with merged capabilities. Our paper explores using heterogeneous model soups to improve LLM alignment by combining models trained with different alignment methods - a novel approach not previously explored in literature. Through empirical evaluation using an “LLM-as-a-judge” approach, we found that mixing different types of models can improve alignment performance, though this requires careful adaptation of interpolation techniques to account for varying alignment objectives. We’ve shared our model merging source code on GitHub ( https://github.com/dawidm/iccs-2025-model-soups ).

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Is Heterogeneous Model Soup Tasty? A Multidimensional Evaluation of Diverse Model Soups in Language Model Alignment

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Dawid Motyka,
  • Paweł Walkowiak,
  • Julia Moska,
  • Bartosz Żuk,
  • Karolina Seweryn,
  • Arkadiusz Janz

摘要

Training and fine-tuning language models is becoming increasingly expensive. “Model soups” offer a promising solution by combining parameters from separately trained models to create a new one with merged capabilities. Our paper explores using heterogeneous model soups to improve LLM alignment by combining models trained with different alignment methods - a novel approach not previously explored in literature. Through empirical evaluation using an “LLM-as-a-judge” approach, we found that mixing different types of models can improve alignment performance, though this requires careful adaptation of interpolation techniques to account for varying alignment objectives. We’ve shared our model merging source code on GitHub ( https://github.com/dawidm/iccs-2025-model-soups ).