Existing Chart Question Answering datasets lack something important: explanations. They provide questions and answers but don’t show the reasoning in between, which limits our ability to build models that genuinely understand charts rather than memorizing patterns. ChartQAR addresses this with 101,554 question-answer-rationale triplets. The dataset makes two key contributions. First, every answer includes step-by-step reasoning designed from the ground up (not added afterwards). Second, it encompasses 14 question types across bar, line, and pie charts—covering basic tasks like counting up through complex multi-hop reasoning. We built this using a three-stage approach: Gemini 2.5 Pro generates candidates, then humans validate and refine them to ensure quality. We tested UniChart on ChartQAR with two training setups. Multi-task learning (treating rationale and answer as separate tasks) achieved 49.51% relaxed accuracy versus 47.00% for single-task learning. Crucially, the error analysis reveals significant disparities: current models struggle significantly with complex reasoning tasks. Multi-step and multi-query questions have error rates of 85–92%, while difference calculations show 76–81% errors. In contrast, perceptual tasks like color and type identification have only 2–34% error rates. Vision-language models still cannot reliably do the math needed for chart understanding. ChartQAR gives researchers the explicit reasoning traces needed to tackle this problem.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ChartQAR: A New Large-Scale Dataset for Complex Reasoning in Chart Question Answering

  • Hoang Khanh Vinh Le,
  • Tam-Hau Kieu,
  • Tuan-Anh Tran,
  • Hong Tai Tran,
  • Xuan Toan Mai

摘要

Existing Chart Question Answering datasets lack something important: explanations. They provide questions and answers but don’t show the reasoning in between, which limits our ability to build models that genuinely understand charts rather than memorizing patterns. ChartQAR addresses this with 101,554 question-answer-rationale triplets. The dataset makes two key contributions. First, every answer includes step-by-step reasoning designed from the ground up (not added afterwards). Second, it encompasses 14 question types across bar, line, and pie charts—covering basic tasks like counting up through complex multi-hop reasoning. We built this using a three-stage approach: Gemini 2.5 Pro generates candidates, then humans validate and refine them to ensure quality. We tested UniChart on ChartQAR with two training setups. Multi-task learning (treating rationale and answer as separate tasks) achieved 49.51% relaxed accuracy versus 47.00% for single-task learning. Crucially, the error analysis reveals significant disparities: current models struggle significantly with complex reasoning tasks. Multi-step and multi-query questions have error rates of 85–92%, while difference calculations show 76–81% errors. In contrast, perceptual tasks like color and type identification have only 2–34% error rates. Vision-language models still cannot reliably do the math needed for chart understanding. ChartQAR gives researchers the explicit reasoning traces needed to tackle this problem.