Improving Remote Sensing Segmentation with Adaptive Tokenization in Vision-Language Models

Semantic segmentation of remote sensing imagery is essential for applications such as land cover mapping, urban monitoring, and water body detection. While Vision-Language Models (VLMs) offer promising zero- and few-shot capabilities, their fixed-length tokenization–originally designed for natural images–fails to capture the irregular, multi-scale structures typical of satellite scenes, leading to coarse boundaries and missed details. We propose an adaptive segmentation tokenization framework that allocates token capacity based on local geometric complexity, computed from perimeter, fractal dimension, and edge density, and encodes regions hierarchically via quadtree decomposition. Integrated with the Paligemma VLM, our method dynamically increases granularity for complex regions while maintaining efficiency in simpler areas. Experiments on three benchmarks–Kaggle Water Bodies, ISPRS Potsdam, and GID–show consistent gains over fixed-token VLMs, improving mIoU by up to +4.5 points (e.g., from 74.0 to 78.5% on water segmentation) and narrowing the gap to fully supervised CNNs by up to 40%. These results demonstrate that content-aware tokenization is a key enabler for accurate and scalable VLM-based remote sensing segmentation.