The increasing adoption of intermittent renewable energy and new prosumer markets, implied the need of more accurate and reliable energy forecasts. In recent researches related large language models (LLMs) opened up new possibilities, they can provide more detailed spatiotemporal analyses of load, and renewable energy predictions. This review analyses recent findings in LLM and its methods such as TimeGPT, STELLM, and BERT4ST that change LLM architectures to work with metadata and time series data. As well as discussing how models are being used along with methods like retrieval augmented generation (RAG), multimodal data fusion, and transfer learning to deal with a lack of data, improve the accuracy of forecasts, to make it easier to get real time operational insights. Furthermore, we enumerate the problems that can come with such large scale neural networks, such as computational cost, data privacy, and the complexity the models. Thus, the main goal of this work is to highlight the use of LLM driven innovations to make energy future more resilient and efficient, and that by providing a critical look at current methods, and pointing out areas to improve, like the sustainability of training and fine tuning for specific domains.