Scholarlink

The increasing demand for large language model (LLM) serving has necessitated significant advancements in the optimization and profiling of LLM inference systems. As these models become integral to a wide range of applications, the need for efficient and scalable serving solutions has grown exponentially. This work introduces TokenSim, a comprehensive hardware and software exploration system designed specifically for LLM inference. TokenSim is characterized by its support for extensible system optimizations including scheduling and memory management. Furthermore, TokenSim facilitates various insightful explorations into the performance and optimization of LLM serving systems. The code is available at https://github.com/pku-lemonade/TokenSim .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

TokenSim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Feiyang Wu,
  • Zhuohang Bian,
  • Guoyang Duan,
  • Tianle Xu,
  • Junchi Wu,
  • Teng Ma,
  • Yongqiang Yao,
  • Ruihao Gong,
  • Youwei Zhuo

摘要

The increasing demand for large language model (LLM) serving has necessitated significant advancements in the optimization and profiling of LLM inference systems. As these models become integral to a wide range of applications, the need for efficient and scalable serving solutions has grown exponentially. This work introduces TokenSim, a comprehensive hardware and software exploration system designed specifically for LLM inference. TokenSim is characterized by its support for extensible system optimizations including scheduling and memory management. Furthermore, TokenSim facilitates various insightful explorations into the performance and optimization of LLM serving systems. The code is available at https://github.com/pku-lemonade/TokenSim .