Radiotherapy treatment planning is one of the most important steps in the radiotherapy treatment workflow. It is responsible for the creation of an individualized treatment for each case, taking into account the characteristics of the volumes to treat and all the surrounding healthy tissues and structures. Typically characterized by a manual, trial-and-error methodology, the selection of irradiation directions and intensities remains a complex endeavor. In this work, a new approach for the automatic generation of radiotherapy treatment plans is described and a proof of concept is considered using prostate cancer cases. In this approach, an unconstrained quadratic optimization problem calculates radiation intensities, with the model parameters being iteratively determined by a fuzzy inference system. To enhance computational efficiency, reinforcement learning, specifically Q-learning, is incorporated. Reinforcement Learning enables the fuzzy inference system to dynamically adjust to the evolving requirements of the iterative optimization process. During the learning phase, Q-tables are constructed and subsequently utilized as lookup tables to guide decision-making in each iteration of the optimization approach. In the conducted computational experiments using cross-validation, a noteworthy reduction of up to 66% in the total number of iterations required to achieve a high-quality treatment plan was observed. While acknowledging the preliminary nature of these results, they unequivocally underscore the promising potential of synergizing Reinforcement Learning and Optimization methodologies in automating the intricate process of radiotherapy treatment planning.