Dynamic Threshold Token Subsampling for Vision Transformer in Cloud Computing
摘要
To address the transmission bottleneck problem of the Vision Transformer (ViT) model caused by high bandwidth demand in edge computing, this paper proposes a dynamic threshold token subsampling method for the Vision Transformer in cloud computing, based on cosine similarity (DCSTS-ViT). The existing subsampling method (CSTS-ViT) based on cosine similarity requires a manually set fixed threshold, which makes the model’s performance sensitive to the dataset and lacks stability. By analyzing the mean value of the cosine similarity matrix and the proportion of high-frequency components of the image, this paper designs a dynamic threshold calculation module: the mean value of cosine similarity measures the degree of semantic repetition, the complexity of the image evaluates by combining the proportion of high-frequency components, and dynamically adjusts the clustering threshold. In the edge device, only the centroid token and interpolation index are transmitted, and the original token matrix is reconstructed by the cloud side using the index, which reduces the amount of transmission data while retaining global semantic information. Experiments on the CIFAR10 and ImageNet100 datasets show that the compression rates of the proposed method are 33.51% and 13.13%, respectively. The accuracy is reduced by only 0.43% and 0.01%, respectively, which significantly outperform the fixed threshold method. In conclusion, our proposal eliminates the need for manual parameter adjustments, thereby enhancing the adaptability and stability of the subsampling model across different datasets, and provides a feasible scheme for the efficient deployment of ViT models in edge-cloud collaborative scenarios.