Compact and efficient language modeling for classical poetry: generation and interpretation
摘要
Chinese classical poetry, as a treasure of Chinese civilization, carries rich historical heritage and unique aesthetic implications. Traditional poetry generation techniques are limited by template-driven constraints, making flexible poetry generation challenging. In recent years, large language models have significantly improved content control flexibility by allowing unrestricted user instructions, but their large number of parameters leads to high computational resource demands and reduced inference efficiency. To overcome this bottleneck and achieve flexible creation in response to instructions, we developed a small-parameter language model, “mini poet”, specifically for the field of Chinese poetry generation and analysis. We introduced a multi-stage training strategy that significantly enhances the model’s generation quality and comprehension depth in the domain of classical poetry through reinforcement with domain-specific data. Specifically, we first built a small Chinese pre-training model using a large-scale general corpus. Subsequently, we meticulously collected and generated a substantial amount of data related to classical poetry to enhance the coverage and diversity of the data. Based on this, we implemented a multi-stage fine-tuning process that effectively optimized the model’s performance in diverse task scenarios, enabling it to better adapt to different types of poetry generation and interpretation tasks. Experimental results show that, through a meticulously designed training process, even with a smaller number of parameters, “Mini Poet” can exhibit performance in classical poetry generation and interpretation that rivals or even surpasses that of large-parameter models.