In the age of digital urban transformation, smart cities are emerging as ecosystems where technology, infrastructure and data converge to improve quality of life, sustainability and economic prosperity. In this context, e-commerce plays a central role, providing platforms for businesses to thrive by enabling seamless transactions, personalised shopping experiences and greater market reach. However, the dynamic and evolving nature of user preferences presents a significant challenge, requiring more adaptive and smart recommendation systems. This paper presents an approach by integrating Deep Q-Network (DQN), a reinforcement learning technique, into recommendation systems for e-commerce in smart cities. By comparing the proposed DQN model based recommender system with traditional models such as MLP, DeepFM, LSTM and CNN using metrics such as MSE, RMSE and NDCG@5, we demonstrate its superior performance in predicting user preferences and dynamically adapting to changes in user behaviour. The results highlight the potential of DQN models to revolutionise e-commerce recommender systems, delivering more personalised and adaptive user experiences in the interconnected environments of smart cities.