Improvement of Diffusion Models Based on Radial Basis Function Neural Networks and Consistency Regularization
摘要
Diffusion models have garnered significant attention in recent years owing to their exceptional performance in image generation and have been progressively explored for applications in text generation. However, despite their outstanding performance in various generative modeling tasks, diffusion models still face numerous challenges when handling discrete data, such as natural language. These challenges include the generation of results that fail to meet expectations, instability, and substantial discrepancies between the generation process and the actual linguistic scenarios. To address these issues, this study proposes a novel framework for enhancing text diffusion model, RadCons-Diffusion, aimed at addressing the issue of semantic inconsistency during the generation process. We propose a synergistic approach: first, a Radial Basis Function Network (RBFN) is employed to construct an embedding space with strong local sensitivity; subsequently, a new form of consistency regularization leverages this well-structured space to ensure that the denoising trajectory remains smooth and semantically coherent. Experimental validation across five distinct control generation tasks demonstrates that RadCons-Diffusion significantly outperforms the existing methods in terms of generation performance.