The energy efficiency of large language models (LLMs) during inference is a critical area of research, driven by the increasing integration of these models across various practical applications. This study examines the energy consumption dynamics of LLMs, focusing on the relationship between batch sizes, model configurations, and energy usage. As demand for computational resources such as GPUs grows, it leads to heightened energy use, raising both environmental and economic concerns. To address these challenges, we leverage the vLLM framework, known for its efficient batch handling, to assess how different batch sizes affect text generation speed and energy consumption. Our findings demonstrate that, while maximizing batch sizes enhances token throughput and reduces energy consumption per token, it can also impact response times. By exploring the use of advanced techniques such as continuous batching, our study provides insights into optimizing LLM deployments for better energy efficiency and sustainability.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Energy Efficiency in Large Language Models: An Empirical Study

  • Tomasz Walkowiak

摘要

The energy efficiency of large language models (LLMs) during inference is a critical area of research, driven by the increasing integration of these models across various practical applications. This study examines the energy consumption dynamics of LLMs, focusing on the relationship between batch sizes, model configurations, and energy usage. As demand for computational resources such as GPUs grows, it leads to heightened energy use, raising both environmental and economic concerns. To address these challenges, we leverage the vLLM framework, known for its efficient batch handling, to assess how different batch sizes affect text generation speed and energy consumption. Our findings demonstrate that, while maximizing batch sizes enhances token throughput and reduces energy consumption per token, it can also impact response times. By exploring the use of advanced techniques such as continuous batching, our study provides insights into optimizing LLM deployments for better energy efficiency and sustainability.