Coloring exercises are a powerful tool for teaching math and computational thinking in young students. But for teachers, grading these tasks is far more complex than checking multiple-choice answers. In earlier work, we suggested using Large Language Models to help with this process, yet current vision models often fail to recognize balls, boxes, or colors when many similar shapes are present. To tackle this, we propose a two-step method: first, we segment the important parts of the image, which include boxes with colored balls, and remove visual distractions, and then we give these cropped sections to a GPT-style model using a zero-shot prompt. This process leads to clear improvements in recognizing colors and objects, even in difficult conditions like poor lighting, shadows, partially colored shapes, and varied marker types—conditions that reflect how kids actually color and how teachers take photos. Our findings show that simple visual pre-processing greatly boosts the accuracy and clarity of Large Vision-Language Models (LVLMs) when used in early education settings.