One of the most common failure types that rapidly deteriorates track geometry is sleeper voids or unsupported sleepers. Ballast pulverization, or so-called “white spots”, is a sign that indicates the presence of the high sleeper voids in the track. However, the objective estimation of the size and form of voids is possible by time- and cost-consuming track-side measurements at many points along the track. The study presents an efficient model-based approach for the identification of the void geometry by the track-side experimental measurements of rail deflection in one point. A robust 3-beam track model with a two-mass vehicle model together with a time-effective surrogate optimization algorithm is used for the multidimensional search of the void geometry that is fitting to experimental data. The results show that the void geometry could be found precisely with one-point measurements, significantly reducing the time and cost involved in the process. Therefore, a practical, simplified method of determining the void zone's depth and length is proposed. It is based on analyzing the relation between void sizes and the rail deflection wave sizes. Unlike the void depth, the void length cannot be found by the simple difference between the deflection waves in the void and the reference zones. The proposed method assumes wave estimation by applying deflection thresholds, ensuring a practical and reliable approach. The reliability of the proposed method in estimating void length and depth instills confidence in the effectiveness of the approach. Finally, it was used to estimate the void length and depth for many problem zones in ballast tracks that were inspected with track-side measurements. The result analysis shows that the void length and depth are subjected to a non-linear relation: the long-length voids have unproportionally higher depths than short voids. The results indicate that the settlement intensity of the neighbor-to-void sleepers is much lower than that of the hanging sleepers.