Human-Aligned GAI Driven by Conceptual Knowledge: System, Framework, and Co-creation

Generative artificial intelligence models (GAI) have become the leading solutions for image generation tasks. Despite their inventive performance, these models suffer from inconsistencies and domain limitations due to data bias and emergent phenomena. Therefore, aligning GAI with human painters has become one of the important interdisciplinary topics of HCI. This paper customizes GAI by embedding domain conceptual knowledge into the large text-to-image model, aiming to learn more about the role of conceptual knowledge in building human-aligned GAI. We first construct a human-AI alignment system from the cognitive perspective. We then propose an alignment method that maps abstract concepts to detailed visual representations, corresponding to a multi-level alignment framework. Finally, we illustrate how the framework can be applied to human-GAI co-creation through experiments on Chinese landscape painting and propose a custom model. The experimental results preliminarily confirm that conceptual knowledge is of great help to building human-aligned GAI. For end users who need a balance between generality and specificity of large models, this study provides the methodology of customizing under the circumstance of limited data and computing power. This study also serves as a valuable reference for researchers to systematically understand human-GAI alignment and advance the alignment of GAI for better adaptation to human-oriented tasks.