Chart-to-code is an emerging task with significant potential in data analysis, automated reporting, and education. It requires accurate visual interpretation of charts and the ability to translate this understanding into executable code. However, existing approaches often struggle to generate precise code for more complex charts, resulting in non-executable code and inaccurate chart reconstructions. To address these challenges, we introduce Visual Sketchbook—a novel framework that employs a multistage optimization process through iterative multimodal feedback, inspired by recent test-time scaling techniques. This design enables more accurate and executable code generation, significantly improving the reliability of the chart-to-code system. Our approach emphasizes a multistage generation process centered on self-reflection and refinement, allowing for progressive reasoning and verification. Experiments show that Visual Sketchbook achieves substantial improvements (on average a 12% gain with a maximum of 17%) in chart-to-code tasks compared to the standard single-pass generation approach. We further demonstrate that the effectiveness and generalizability of our proposed approach through detailed analysis and ablation studies.