Scholarlink

Large Language Models (LLMs) have revolutionized artificial intelligence, showcasing unprecedented capabilities across a wide range of tasks and significantly enhancing edge computing. However, deploying LLMs on edge devices presents challenges due to the distributed nature of users across edge nodes and the complexity of request types. To address these challenges, this paper first establishes an LLM inference model within an edge computing system and formulates an optimization problem aimed at maximizing the number of requests processed per unit time while satisfying user demands. To solve this problem, we model the system as a Semi-Markov Decision Process (SMDP) and employ reinforcement learning methods to propose the Dynamic Inference Separable Proximal Policy Optimization (DISPPO) algorithm. This algorithm dynamically adjusts the order of the prefill and decode stages. We evaluate the performance of DISPPO in edge-based LLM inference by comparing it with traditional inference methods. Simulation results demonstrate that DISPPO outperforms traditional methods in LLM inference scenarios. Additionally, we explore the algorithm’s performance across different batch sizes.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Adaptive Batch Processing and Resource Allocation for LLM Inference in Edge Computing

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Lei Yang,
  • Shuying Gan,
  • Xijun Wang,
  • Jing Liu,
  • Xiang Chen

摘要

Large Language Models (LLMs) have revolutionized artificial intelligence, showcasing unprecedented capabilities across a wide range of tasks and significantly enhancing edge computing. However, deploying LLMs on edge devices presents challenges due to the distributed nature of users across edge nodes and the complexity of request types. To address these challenges, this paper first establishes an LLM inference model within an edge computing system and formulates an optimization problem aimed at maximizing the number of requests processed per unit time while satisfying user demands. To solve this problem, we model the system as a Semi-Markov Decision Process (SMDP) and employ reinforcement learning methods to propose the Dynamic Inference Separable Proximal Policy Optimization (DISPPO) algorithm. This algorithm dynamically adjusts the order of the prefill and decode stages. We evaluate the performance of DISPPO in edge-based LLM inference by comparing it with traditional inference methods. Simulation results demonstrate that DISPPO outperforms traditional methods in LLM inference scenarios. Additionally, we explore the algorithm’s performance across different batch sizes.