Commonsense visual sensemaking requires robust mechanisms to extract scene elements and other visual features from perceived imagery, as well as rich conceptual commonsense knowledge and reasoning to interpret the perceived interactive dynamic. In this context, we position ongoing work on using integrated Vision Language Models (VLM) and Answer Set Programming (ASP) based reasoning about (dynamic) spatial configurations using VLMs as a neural foundation for symbolic modeling of dynamic scene structures. We sketch preliminary results highlighting practical usability by application to the task of Visual Question Answering (VQA) with the STRIDE-QA driving dataset.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Towards a VLM-Based Foundation for Generalised Neurosymbolic Visual Commonsense

  • Jakob Suchan,
  • Salim Baloch,
  • Mehul Bhatt

摘要

Commonsense visual sensemaking requires robust mechanisms to extract scene elements and other visual features from perceived imagery, as well as rich conceptual commonsense knowledge and reasoning to interpret the perceived interactive dynamic. In this context, we position ongoing work on using integrated Vision Language Models (VLM) and Answer Set Programming (ASP) based reasoning about (dynamic) spatial configurations using VLMs as a neural foundation for symbolic modeling of dynamic scene structures. We sketch preliminary results highlighting practical usability by application to the task of Visual Question Answering (VQA) with the STRIDE-QA driving dataset.