Neuro-Symbolic Reasoning with Multiple Large Language Models Combined by First-Order Logic
摘要
Neuro-symbolic reasoning aims to integrate perceptual understanding with structured logical inference. However, most existing approaches rely on rigid symbolic conversion or supervised training pipelines, limiting their ability to harness the rich, implicit knowledge embedded in pre-trained models. We propose a training-free neuro-symbolic model that integrates multiple large language models (LLMs) by converting their latent knowledge into unified first-order logic (FOL) representations. Reasoning is grounded in these logic forms with natural language that provides an interpretable reasoning process. Given an input, the proposed model first parses object-centric features, invokes an LLM to generate structured FOL-based representation, and deduces answers through language-based logical inference—without any task-specific learning, program execution, or external symbolic modules. The proposed model achieves superior results across diverse visual QA benchmarks: It improves performance over LLaVA by 42%p for global queries in GQA and surpasses BLIP2 by 19.4%p for OK-VQA and by 0.4%p for A-OKVQA. These results demonstrate the effectiveness of logic-based knowledge externalization as a foundation for interpretable and generalizable visual reasoning.