The optimization of Structured Query Language (SQL) Queries is one of the factors that influence the performance of the database system but more so at low-end computers where the efficient use of resources is critical. Most of the Query optimization methods implemented today are static; that is, they use fixed heuristics to improve the query irrespective of the changes in demand of the system. The major contribution of this work is the enhancement of Query optimization using Reinforcement Learning techniques such as Deep-Q-Networks (DQN), Double Deep-Q-Networks (DDQN) and Proximal Policy Optimization (PPO) algorithms. PPO is a much better choice for continuous action spaces whereas for the discrete SQL Environment, DQN and DDQN perform better. The choice of using DDQN over DQN has been made as DDQN is more stable and reduces the overestimation bias and Q-Learning update. All these techniques aim to optimize the Execution Time, CPU Usage, Memory Usage, Query Complexity, System Load. Additionally, the RL environment also considers Bandwidth Usage, which is essential for any Edge Devices connected to the Gateway. The reward proposed in our work encourages the agent to take successful actions that fulfill the agent’s goals. This paper also narrates about the deployment of the saved RL model in an Edge Device (Raspberry PI) using Model Reduction Techniques like pruning and quantization. Thus, the hardware deployment of the proposed DDQN based on light weight SQL optimization model exhibits power saving of around 30%.