Scholarlink

<p>There have been significant advancements in the development of large language models (LLMs). Models such as GPT, LLAMA, and Mistral demonstrate the ability to comprehend real-world concepts, as they are trained on extensive datasets that include diverse real-world information. In this study, we propose leveraging the information encoding capabilities of LLMs to address the real-world problem of electricity forecasting. The patterns of electricity consumption in residential buildings are significantly influenced by weather conditions, particularly due to heating and cooling demands. The daily variability caused by these weather conditions presents a challenge for accurate short-term forecasting. Recent implementations of LLMs have shown their potential in capturing broader contextual information through advanced encoding mechanisms. This paper explores an innovative approach to effectively utilize LLMs for encoding weather-related information to enhance forecasting accuracy. Specifically, we employ two large language models—CLIP and BERT—and demonstrate how encoding weather information using a predefined caption format, followed by extracting embeddings from these captions using LLMs, results in superior electricity forecasting performance. Our empirical results show that this approach leads to significantly lower RMSE and MAE values compared to traditional SLSTM models, and the CLIP model achieves an average inference time of just 0.75 milliseconds per 3-min prediction.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Short-Term Residential Load Forecasting Using Textual Embeddings-Based Multi-encoder Transformer

  • 期刊论文

  • 发表日期：

    2025年10月13日

查看全文
  • Vangipuram Srinivasa Raghavan,
  • A. V. Giridhar

摘要

There have been significant advancements in the development of large language models (LLMs). Models such as GPT, LLAMA, and Mistral demonstrate the ability to comprehend real-world concepts, as they are trained on extensive datasets that include diverse real-world information. In this study, we propose leveraging the information encoding capabilities of LLMs to address the real-world problem of electricity forecasting. The patterns of electricity consumption in residential buildings are significantly influenced by weather conditions, particularly due to heating and cooling demands. The daily variability caused by these weather conditions presents a challenge for accurate short-term forecasting. Recent implementations of LLMs have shown their potential in capturing broader contextual information through advanced encoding mechanisms. This paper explores an innovative approach to effectively utilize LLMs for encoding weather-related information to enhance forecasting accuracy. Specifically, we employ two large language models—CLIP and BERT—and demonstrate how encoding weather information using a predefined caption format, followed by extracting embeddings from these captions using LLMs, results in superior electricity forecasting performance. Our empirical results show that this approach leads to significantly lower RMSE and MAE values compared to traditional SLSTM models, and the CLIP model achieves an average inference time of just 0.75 milliseconds per 3-min prediction.