Automated diagnosis is gaining increasing attention as it can assist human doctors in gathering patient information and diagnosing diseases. Currently, generative methods, particularly those based on Transformer models, have achieved significant success in the field of automated diagnosis. However, our method in this paper brings confidence to the application of reinforcement learning (RL) in this domain. Traditional RL methods have struggled to perform well on medical datasets mainly due to the highly imbalanced distribution of disease symptoms. To address this, we propose a machine experience-based reinforcement learning diagnostic framework (MERLDF). Specifically, we use the distribution of relevant data generated by the agent during training as machine experience and combine action and reward reshaping to aid the agent’s learning process. This approach effectively enhances the performance of RL on medical diagnosis datasets. Our experiments were conducted on four real datasets. The results show that, compared to state-of-the-art methods, our method improves the overall performance (score) by 1.9% and 2.6% for maximum dialogue turns of 10 and 20, respectively. The code is available at: https://github.com/H-F-Liang/MERLDF .