Controllable text generation is currently a hot research topic in artificial intelligence. It involves manipulating the semantics and style of generated text by applying various constraints. Typically, this is done by combining hard constraints, such as ensuring specific keywords are included, with soft constraints, like maintaining coherence with the surrounding context. However, these methods often fall short in fully controlling the diversity, fluency, and naturalness of the generated text. Moreover, the dynamic adjustment of constraints due to changing application scenarios requires a large amount of annotated data, which is costly. To address this, this paper proposes a novel text generation framework based on large language models and Hamiltonian systems. In this framework, text generation is treated as a dynamic system, where the step-by-step construction of text (e.g., from words to sentences) is analogous to the evolution of the system's state. The framework employs the Hamiltonian function \(H\left( {q,p} \right)\) from Hamiltonian mechanics as a constraint. Here, the kinetic energy component of the Hamiltonian function acts as a control vector to regulate text diversity, while the potential energy component enforces constraints to ensure text fluency and naturalness. The system's state evolution is dynamically fine-tuned by integrating large language models with COLD decoding methods. This method is particularly applied to the challenging task of abductive reasoning in text generation. Experimental results confirm the effectiveness of the proposed approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Research on Controllable Text Generation Method Based on Hamiltonian Systems and Large Language Models

  • Delong Xu,
  • Min Lin,
  • Yurong Wang

摘要

Controllable text generation is currently a hot research topic in artificial intelligence. It involves manipulating the semantics and style of generated text by applying various constraints. Typically, this is done by combining hard constraints, such as ensuring specific keywords are included, with soft constraints, like maintaining coherence with the surrounding context. However, these methods often fall short in fully controlling the diversity, fluency, and naturalness of the generated text. Moreover, the dynamic adjustment of constraints due to changing application scenarios requires a large amount of annotated data, which is costly. To address this, this paper proposes a novel text generation framework based on large language models and Hamiltonian systems. In this framework, text generation is treated as a dynamic system, where the step-by-step construction of text (e.g., from words to sentences) is analogous to the evolution of the system's state. The framework employs the Hamiltonian function \(H\left( {q,p} \right)\) from Hamiltonian mechanics as a constraint. Here, the kinetic energy component of the Hamiltonian function acts as a control vector to regulate text diversity, while the potential energy component enforces constraints to ensure text fluency and naturalness. The system's state evolution is dynamically fine-tuned by integrating large language models with COLD decoding methods. This method is particularly applied to the challenging task of abductive reasoning in text generation. Experimental results confirm the effectiveness of the proposed approach.