Accurate active speaker detection is essential for natural verbal human-robot interaction. The available solutions have mainly focused on audio-based Voice Activity Detection (VAD), but these approaches become insufficient when audio is compromised or unavailable due to contextual factors. In such cases, the robot must infer whether someone is speaking using only video input. This paper establishes a foundation to enhance robot multimodal dialogue systems by integrating Visual Voice Activity Detection (VVAD) into the social robot Haru. Unlike prior studies that focus on detecting isolated speech segments, our method shifts toward accurately identifying speech boundaries, enabling the robot to handle turns from a visual perspective. We propose new metrics that better capture VVAD behavior in dynamic turn-taking scenarios, as well as overall speech and silence detection. Our results, which align with state-of-the-art benchmarks in isolated segments, highlight the effectiveness of VVAD in accurately identifying relevant speech instances with a simple expansion of a chunk-based algorithm to mark turns. These findings indicate the feasibility of incorporating VVAD into vision-based robotics and encourage further exploration in real-world applications to address remaining challenges in speech-based human-robot interaction, where visual detection is often the most reliable—if not the only—method for identifying speakers.