This study presents a comprehensive comparative analysis of electricity consumption patterns of USA and India, using a variety of basic and advanced Machine Learning (ML) techniques. ML can accurately predict and forecast the energy demand, allowing for more efficient management of energy grids. Herein, we utilized multiple basic and advanced ML models, including XGBoost, Convolutional Neural Networks, Long Short-Term Memory (LSTM) networks, Transformer models, and Reinforcement Learning (RL) for electricity demand modeling. All ML models were evaluated for accuracy of electricity consumption prediction over short-term and long-term periods. Additional datasets of climate, economic indicators, and renewable energy metrics were also incorporated to enhance the prediction accuracy. Results show that the Transformer models outperformed all other models, due to their ability to accurately capture long-range temporal dependencies for both countries. LSTM networks also demonstrated robust performance, particularly for India, where frequent electricity demand fluctuations are more prevalent. XGBoost performed well for short-term predictions, owing to its capability to handle non-linear relationships between the drivers. Furthermore, RL provided an innovative solution for electricity usage planning and management. This paper presents key insights for energy policymakers and companies on improving grid efficiency, integrating renewable energy sources, future planning and pricing policies.