To address the issues of the Particle Swarm Optimization (PSO) algorithm, such as being prone to getting trapped in local optima, having low solution accuracy, and insufficient search in high-dimensional or large search spaces, an improved PSO algorithm integrating simulated annealing and reinforcement learning is proposed. Firstly, the particle update mechanism in the PSO algorithm is employed to generate the initial solutions, and the temperature parameter and update rules of the Simulated Annealing (SA) algorithm are utilized to further adjust the search range, thus avoiding the algorithm from falling into local optima. Subsequently, the Q-table and exploration rate mechanism of the Reinforcement Learning (RL) algorithm are combined to guide the particles to adjust their action strategies based on the state-action values during the iteration process, thereby balancing exploration and exploitation. The PSA-RL algorithm can conduct global search using PSO, avoid local optima with the help of the temperature mechanism of SA, and further optimize the search strategy through the adaptive learning of RL, enhancing the global optimization ability and convergence speed. Five test functions are selected to simulate the performance of the algorithm. The simulation results demonstrate that the PSA-RL algorithm has significant performance advantages compared to the Standard PSO, Local Best PSO (L-Best PSO), and Adaptive PSO (A-PSO), proving that the proposed PSA-RL algorithm has strong optimization capabilities.