Bongard Problems (BPs) pose a fundamental challenge in the abstract visual reasoning (AVR) domain. To solve a BP, one has to identify an abstract pattern that differentiates images from the left and right sides of the matrix, which requires strong perception and reasoning capabilities. In this paper, we investigate whether vision language models (VLMs) can solve BPs by considering three binary classification problem settings involving both vision and text modalities. Our experiments primarily involve 4 open-weight models including InternVL2, LLaVa-1.6, Phi 3.5 V, and Pixtral. We employ 4 BP datasets involving synthetic and real-world images: classic BPs, Bongard HOI, Bongard-OpenWorld, and Bongard-RWR. The conducted experiments show that certain models perform better in a decoupled setting, where panel descriptions are provided before-hand, suggesting their limitations in processing the vision modality. In contrast, other models achieve strong results in a uniform setup, which doesn’t involve an explicit panel description step, indicating their strength in bi-modal reasoning. We conclude by summarizing emerging research directions in the field to further advance abstract reasoning capabilities of VLMs.