Recent advancements in large-scale language models (LLMs) have significantly transformed automatic question answering (QA) systems, achieving remarkable performance improvements. While pre-trained LLMs demonstrate strong cross-domain adaptability, their effectiveness in specialized domains remains inconsistent. Furthermore, the computational costs associated with large-parameter models often necessitate the deployment of smaller-scale models (with several billion parameters) in practical applications, despite their compromised QA accuracy. To address these challenges, this study introduces the Automatic Question Answering Framework based on LLM Collaboration (AQFM). AQFM integrates model selection with iterative reasoning through DQN-based reinforcement learning. By strategically defining action and state spaces, a policy model is trained to dynamically optimize questioning strategies in multi-round, multi-model scenarios. Experimental results demonstrate that AQFM achieves a 1.07 and 1.38 improvement in mean task ranking across diverse QA scenarios, significantly enhancing QA accuracy. Compared to single-model approaches and traditional multi-model collaboration methods, AQFM outperforms existing solutions across multiple datasets and evaluation metrics. By effectively enhancing the performance of smaller-parameter models in domain-specific tasks, AQFM provides a scalable and cost-effective solution for real-world automatic QA systems.