Managing large-scale electrical power grids is crucial for ensuring a stable and reliable energy supply. Smart grids, utilizing advanced algorithms, are proposed as a solution, with Reinforcement Learning (RL) playing a key role. This paper explores the application of RL in training an agent to manage a simulated power grid using the City Learn simulator. However, the extended training times pose practical challenges. I discuss these challenges and alternative strategies to expedite the training process, shedding light on the complexities of implementing RL in real-world power grid management.