In the medical field, HuatuoGPT-o1, as the first medical large language model (LLM) capable of complex reasoning, has demonstrated outstanding performance on multiple medical datasets that require reasoning. However, the chain-of-thought (CoT) process in HuatuoGPT-o1 generates thousands of tokens, resulting in a significant demand for computational resources and time. Our analysis reveals that these tokens contribute differently to the final answer, thus leading to the proposal of the MedSparse method. MedSparse focuses on the key steps in the CoT process, enabling HuatuoGPT-o1 to better understand the importance of these steps in the reasoning process. Unlike HuatuoGPT-o1, which relies on the final answer as a supervisory signal, MedSparse uses the key steps in CoT as the supervisory signal, allowing HuatuoGPT-o1 to learn more effectively the role of these key steps in reasoning. MedSparse compresses the CoT in a controlled manner, optimizing it in three aspects: case description, background information, and logical reasoning. Experimental results show that MedSparse significantly reduces token usage while maintaining strong reasoning performance. Specifically, when the token count is reduced to half of the original amount, reasoning speed increases by 1.76 times, while performance remains at 93% of the original. Compared with various 7B-scale general models and medical models, MedSparse consistently outperforms other models on multiple medical datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MedSparse: A Medical Large Model for Efficient Inference and Chain-of-Thought Generation

  • Yue Zhu,
  • Dengke Deng,
  • Ya Li,
  • Xiao’er Li,
  • Zhuo Li,
  • Pengcheng Luo

摘要

In the medical field, HuatuoGPT-o1, as the first medical large language model (LLM) capable of complex reasoning, has demonstrated outstanding performance on multiple medical datasets that require reasoning. However, the chain-of-thought (CoT) process in HuatuoGPT-o1 generates thousands of tokens, resulting in a significant demand for computational resources and time. Our analysis reveals that these tokens contribute differently to the final answer, thus leading to the proposal of the MedSparse method. MedSparse focuses on the key steps in the CoT process, enabling HuatuoGPT-o1 to better understand the importance of these steps in the reasoning process. Unlike HuatuoGPT-o1, which relies on the final answer as a supervisory signal, MedSparse uses the key steps in CoT as the supervisory signal, allowing HuatuoGPT-o1 to learn more effectively the role of these key steps in reasoning. MedSparse compresses the CoT in a controlled manner, optimizing it in three aspects: case description, background information, and logical reasoning. Experimental results show that MedSparse significantly reduces token usage while maintaining strong reasoning performance. Specifically, when the token count is reduced to half of the original amount, reasoning speed increases by 1.76 times, while performance remains at 93% of the original. Compared with various 7B-scale general models and medical models, MedSparse consistently outperforms other models on multiple medical datasets.