Enhancing Chinese comprehension and reasoning for large language models: an efficient LoRA fine-tuning and tree of thoughts framework

Large language models have been extensively utilized for addressing general problems across various tasks. However, their current implementation in large language models still exhibits constraints in Chinese comprehension and reasoning. This study aims to overcome these limitations through enhancing the understanding and reasoning capabilities of Chinese language models while minimizing resource demands. We introduce LLaMA-LoRA, an engineered framework driven by thoughtful guidance, which is constructed upon the LLaMA-13B model and incorporates low-rank adaptation (LoRA) for meticulous fine-tuning. To overcome the inherent constraint of token-based left-to-right decision making during the inference process, we introduce the tree of thought to further augment our model. By considering diverse reasoning paths and engaging in self-evaluation during the decision-making process, Tree of thought facilitates deliberate decision making and facilitates the determination of subsequent action plans. Our experimental results demonstrate that the integration of this strategic mechanism significantly enhances Chinese comprehension and reasoning abilities, surpassing the performance of GPT-3.5, Chatglm, and other relevant language models in specific tasks. Meanwhile, it demonstrates excellent performance in sentiment calculation tasks such as multiclass sentiment analysis. Moreover, the availability of our open-source code facilitates further advancements in the field of Chinese text logical inference.