Reinforcement Learning (RL) is a type of machine learning technique in which an intelligent autonomous agent makes observations in an environment. The agent receives rewards or punishments, in order to maximize cumulative rewards through trial and error. RL has made significant achievements in the fields of robotics, economics, finance, and autonomous cars. In the context of urbanism, growing populations have complicated the task of urban planning, increasing the need to optimize cities for livability. An approach to achieving this is through the n-minute city model, which ensures that all essential services and facilities are located within a short distance of every residence. The purpose of this investigation was to build an agent capable of designing a city that maximized livability by optimizing the interactions between buildings and urban infrastructure. The RL framework was used because it provides tools for programming an agent to design the city without explicitly defining its layout. Using this approach raises questions about whether RL is the best method, which hyperparameters are optimal, the trade-off between exploration and exploitation, and the comparison between Q-learning and Deep Q-learning. This paper outlines how this methodology can be applied and the advantages of this approach in urban planning.