VTQA: visual tabular data question answering framework based on multi-agent
摘要
With the rapid development of large language models (LLMs), natural language understanding and reasoning capabilities of tabular question answering (TQA) systems have significantly improved. However, existing approaches often suffer from invisible intermediate processes, uncontrollable execution paths, and limited user involvement, making them inadequate for complex real-world analytical tasks that demand interpretability and interactivity. To address these challenges, we propose VTQA (visual tabular question answering), a novel framework that integrates multi-agent logical reasoning mechanisms with visualization techniques, using structured instructions as the core to bridge user semantics, data objects, and execution flows. VTQA consists of key components including object structure extraction, semantic parsing, chained instruction generation, execution, and feedback. It adopts four-layer architecture—object layer, semantic layer, instruction layer, and visualization layer—to ensure structural transparency and semantic controllability, supporting user intervention and iterative refinement. We also design an interactive visualization system that systematically integrates intermediate execution results and query flows, offering feedback channels to users. Three case studies based on real-world tabular datasets and user studies demonstrate the effectiveness and usefulness of our system.
Graphical abstract