<p>In this paper, a hybrid convolutional neural network (CNN) and transformer architecture for image classification that explicitly exploits multiscale spatial representations while maintaining computational efficiency is proposed. A convolutional backbone is first used to extract hierarchical feature maps, which are subsequently tokenized and fused through a cross-scale token fusion (CSTF) mechanism. In addition, several token-level and CNN-level pruning strategies are evaluated to examine whether redundant spatial tokens or convolutional features can be removed without substantially degrading performance. Extensive experiments on Caltech-101 and Oxford-IIIT Pets under low-data training conditions show that the best proposed configurations are dataset-dependent: the single 14<InlineEquation ID="IEq1"><EquationSource Format="TEX">\(\:\times\:14\)</EquationSource></InlineEquation> scale achieves 70.28<InlineEquation ID="IEq2"><EquationSource Format="TEX">\(\:\pm\:1.91\)</EquationSource></InlineEquation>% accuracy on Caltech-101, while the single 7<InlineEquation ID="IEq3"><EquationSource Format="TEX">\(\:\times\:\)</EquationSource></InlineEquation>7 scale achieves 28.35<InlineEquation ID="IEq4"><EquationSource Format="TEX">\(\:\pm\:1.62\)</EquationSource></InlineEquation>% accuracy on Oxford-IIIT Pets. Among the multi-scale fusion models, 7<InlineEquation ID="IEq5"><EquationSource Format="TEX">\(\:\times\:\)</EquationSource></InlineEquation>7+14<InlineEquation ID="IEq6"><EquationSource Format="TEX">\(\:\times\:\)</EquationSource></InlineEquation>14 performs best on Caltech-101 69.68 <InlineEquation ID="IEq7"><EquationSource Format="TEX">\(\:\pm\:\)</EquationSource></InlineEquation>1.77%, whereas 7<InlineEquation ID="IEq8"><EquationSource Format="TEX">\(\:\times\:\)</EquationSource></InlineEquation>7+14<InlineEquation ID="IEq9"><EquationSource Format="TEX">\(\:\times\:\)</EquationSource></InlineEquation>14+28<InlineEquation ID="IEq10"><EquationSource Format="TEX">\(\:\times\:\)</EquationSource></InlineEquation>28 performs best on Oxford-IIIT Pets 27.57<InlineEquation ID="IEq11"><EquationSource Format="TEX">\(\:\pm\:\)</EquationSource></InlineEquation>1.05%. CNN kernel pruning achieves the strongest pruning performance on both datasets, reaching 68.37<InlineEquation ID="IEq12"><EquationSource Format="TEX">\(\:\pm\:\)</EquationSource></InlineEquation>1.06% on Caltech-101 and 28.15<InlineEquation ID="IEq13"><EquationSource Format="TEX">\(\:\pm\:\)</EquationSource></InlineEquation>1.15% on Oxford-IIIT Pets. These results indicate that multi-scale token fusion can provide competitive performance, but careful scale selection is more important than simply increasing the number of spatial token scales.</p>

Application of multiscale token fusion and pruning in CNN–Transformer hybrids under low-data training for image recognition

期刊论文
出版模式：

开放获取
发表日期：

2026年7月25日

查看全文

Yuan Cao

摘要

In this paper, a hybrid convolutional neural network (CNN) and transformer architecture for image classification that explicitly exploits multiscale spatial representations while maintaining computational efficiency is proposed. A convolutional backbone is first used to extract hierarchical feature maps, which are subsequently tokenized and fused through a cross-scale token fusion (CSTF) mechanism. In addition, several token-level and CNN-level pruning strategies are evaluated to examine whether redundant spatial tokens or convolutional features can be removed without substantially degrading performance. Extensive experiments on Caltech-101 and Oxford-IIIT Pets under low-data training conditions show that the best proposed configurations are dataset-dependent: the single 14\(\:\times\:14\) scale achieves 70.28\(\:\pm\:1.91\)% accuracy on Caltech-101, while the single 7\(\:\times\:\)7 scale achieves 28.35\(\:\pm\:1.62\)% accuracy on Oxford-IIIT Pets. Among the multi-scale fusion models, 7\(\:\times\:\)7+14\(\:\times\:\)14 performs best on Caltech-101 69.68 \(\:\pm\:\)1.77%, whereas 7\(\:\times\:\)7+14\(\:\times\:\)14+28\(\:\times\:\)28 performs best on Oxford-IIIT Pets 27.57\(\:\pm\:\)1.05%. CNN kernel pruning achieves the strongest pruning performance on both datasets, reaching 68.37\(\:\pm\:\)1.06% on Caltech-101 and 28.15\(\:\pm\:\)1.15% on Oxford-IIIT Pets. These results indicate that multi-scale token fusion can provide competitive performance, but careful scale selection is more important than simply increasing the number of spatial token scales.