Inheritance and Revitalization: Exploring the Synergy Between AIGC Technologies and Chinese Traditional Culture

Diffusion models like Stable Diffusion have made impressive progress in T2I (text-to-image) generation. However, when applied to image generation tasks concerned with Chinese cultural subjects, Stable Diffusion needs to improve the quality of its results. This paper proposes a practical approach to address the challenges of utilizing popular AIGC (AI-Generated Content) technologies and integrating them into a cohesive system, which makes it easier to use Stable Diffusion to create high-quality generated images related to Chinese cultural subjects with direct Chinese prompts. Specifically, with the capabilities of Large Language Models (LLMs), the approach can weave expressive visual descriptions based on initial inputs (scattered words in Chinese, Chinese poems…) and align them in suitable English text prompts for subsequent image generation with Stable Diffusion. Through the parameter-efficient finetuning method called LoRA, Stable Diffusion can effectively learn complex and nuanced concepts of Chinese culture. Additionally, Prompt Engineering plays a role in optimizing inputs, assuring quality and stability, and setting the detailed behavior patterns of LLMs throughout the workflow. This success is attributed to overcoming the constraints of accepting only English prompts and significantly improving the understanding of certain concepts in Chinese culture. The experiments show that our method can produce high-quality images associated with complex and nuanced concepts in Chinese culture by leveraging the fusion of all independent components.