S2Q: Teaching Language Models New Facts Through Knowledge Graph Instruction Synthesis
摘要
Instruction tuning has revolutionized large language models (LLMs), yet their development remains bottlenecked by the need for extensive human-annotated data. While synthetic data generation methods like Self-Instruct attempt to address this limitation, they merely recycle knowledge already embedded in pre-trained models, failing to inject genuinely new information. We present Structure-to-Question (S2Q), a novel framework that transforms knowledge graphs (KGs) into high-quality instruction data without human annotation. S2Q leverages a key insight: the topological structure of KGs—from single edges to multi-hop paths—naturally corresponds to question complexity patterns, from simple factual queries to complex reasoning tasks. Our pipeline systematically samples subgraphs, converts structural patterns into natural language questions via LLM prompting, and ensures linguistic diversity through controlled generation, all while maintaining rigorous answer validation to prevent hallucination. Unlike existing approaches that use KGs solely for retrieval or verification, S2Q directly teaches structured knowledge to model parameters. Experiments demonstrate that Llama-3.1-8B models trained exclusively on S2Q-generated data achieve 78.5% accuracy on WebQSP and 42.1% on ComplexWebQuestions—approaching supervised baselines without using any human-labeled examples. By reimagining knowledge graphs as autonomous instructors rather than passive databases, S2Q opens new avenues for scalable, knowledge-grounded LLM training, particularly benefiting domains where structured knowledge is abundant but human annotations are scarce.