Text-to-SQL is designed to translate natural language questions into corresponding SQL statements. A common approach involves fine-tuning a large language model (LLM) on a limited labeled dataset. However, this approach often has difficulty handling complex queries and generalizing to other scenarios. To bridge this gap, instead of relying solely on manually annotated data, we use LLM to gradually generate high-quality, diverse data based on natural language cues and database schemas. This synthetic data generation can be viewed as a form of knowledge distillation. In addition, we further enrich the training data through strategies such as semantic rewriting, numerical condition conversion, and query logic modification. To ensure data quality and relevance, a multi-level filtering mechanism consisting of syntax verification and semantic consistency checking is employed to identify and eliminate low-quality samples. The effectiveness of the proposed method is evaluated on the Spider benchmark dataset and its extended version, and the results show that our approach improves the generalization and robustness of the model.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LLM-Based Data Synthesis and Distillation for High-Quality Text-to-SQL Training

  • Shaoguo Cui,
  • Keying Wen,
  • Binbin Sang,
  • Tiansong Li,
  • Yi Zhang,
  • Huan Gao

摘要

Text-to-SQL is designed to translate natural language questions into corresponding SQL statements. A common approach involves fine-tuning a large language model (LLM) on a limited labeled dataset. However, this approach often has difficulty handling complex queries and generalizing to other scenarios. To bridge this gap, instead of relying solely on manually annotated data, we use LLM to gradually generate high-quality, diverse data based on natural language cues and database schemas. This synthetic data generation can be viewed as a form of knowledge distillation. In addition, we further enrich the training data through strategies such as semantic rewriting, numerical condition conversion, and query logic modification. To ensure data quality and relevance, a multi-level filtering mechanism consisting of syntax verification and semantic consistency checking is employed to identify and eliminate low-quality samples. The effectiveness of the proposed method is evaluated on the Spider benchmark dataset and its extended version, and the results show that our approach improves the generalization and robustness of the model.