Fine-Tuning for Low-Resource Language Machine Translation Using Large Language Models Integrated with Dependency Parsing Rule
摘要
The loss of endangered languages is a severe threat to cultural heritage. Machine translation holds significant promise for documenting and preserving these languages. However, low-resource languages, such as Tujia, which lacks both parallel corpora and a standardized written form, pose substantial challenges for machine translation. In order to explore machine translation methods for low-resource language without written text, this paper takes Tujia as an example. First, we analyzed the grammatical and lexical correlations between Tujia and Chinese, constructing a Tujia dependency parsing treebank and structuring parsing knowledge. Second, in view of the low-resource and non-literate characteristics of Tujia, we applied knowledge transferred from Large Language Models (LLMs), designed prompt engineering, and fine-tuned the Llama and ChatGLM models. Dynamic dataset partitioning is employed, and translation quality is assessed using Translation Edit Rate (TER). Results indicated that the structured fine-tuning approach based on ChatGLM performed best, with LoRA fine-tuning yielding the most effective results. Incorporating dependency parsing rule into the structured instruction set significantly outperformed unstructured instruction fine-tuning, effectively mitigating the loss of grammatical information in unwritten languages and enhancing translation quality. Finally, we utilized the Llama. cpp framework to accelerate inference and improve model performance and practicality.