<p>2D convolutional neural networks (CNNs) achieve remarkable accuracy across diverse computer vision tasks, yet their inference efficiency remains suboptimal. As a fast convolution algorithm, Winograd convolution can significantly accelerate convolution operations, which constitute the primary performance bottleneck in CNNs. However, existing GPU-based implementations are restricted to accelerating standard convolution with unit dilation, while lacking support for dilated convolution. We propose TC-DWC, a GPU-based 2D Winograd convolution implementation tailored for dilated convolution, which effectively accelerates dilated convolution tasks across various configurations. TC-DWC employs a two-step tile reorganization scheme that reconciles the incompatibility between Winograd convolution and dilated convolution. Moreover, it leverages specialized high-throughput tensor cores (TCs), replacing conventional vector units to accelerate the computationally dominant matrix multiplications. To alleviate TC-DWC’s substantial memory footprint, we further develop a multi-stage kernel fusion strategy that eliminates intermediate array allocations, yielding Fused TC-DWC. Experimental results demonstrate that, for single convolutional layers, TC-DWC and Fused TC-DWC deliver average speedups of <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$1.56\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.56</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation> (up to <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$2.45\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>2.45</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation>) and <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$1.29\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.29</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation> (up to <InlineEquation ID="IEq4"> <EquationSource Format="TEX">$1.80\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.80</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation>) over cuDNN optimal implementation, respectively, with Fused TC-DWC reducing memory footprint to 35% of TC-DWC and 55% of cuDNN. For end-to-end inference of CSRNet and DLinkNet34, TC-DWC and Fused TC-DWC deliver average speedups of <InlineEquation ID="IEq5"> <EquationSource Format="TEX">$1.38\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.38</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation> (up to <InlineEquation ID="IEq6"> <EquationSource Format="TEX">$1.41\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.41</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation>) and <InlineEquation ID="IEq7"> <EquationSource Format="TEX">$1.20\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.20</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation> (up to <InlineEquation ID="IEq8"> <EquationSource Format="TEX">$1.22\times $</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.22</mn> <mo>×</mo> </mrow> </math></EquationSource> </InlineEquation>) over PyTorch baseline, respectively, with Fused TC-DWC enabling support for larger batch sizes.</p>

Accelerating dilated Winograd convolution with fused GPU kernel using tensor cores

期刊论文
出版模式：

开放获取
发表日期：

2026年3月16日

查看全文

Shixiang Zhang,
Jianguo Liang,
You Fu,
Rong Hua,
Jianzhi Yu,
Qianqian Li

摘要

2D convolutional neural networks (CNNs) achieve remarkable accuracy across diverse computer vision tasks, yet their inference efficiency remains suboptimal. As a fast convolution algorithm, Winograd convolution can significantly accelerate convolution operations, which constitute the primary performance bottleneck in CNNs. However, existing GPU-based implementations are restricted to accelerating standard convolution with unit dilation, while lacking support for dilated convolution. We propose TC-DWC, a GPU-based 2D Winograd convolution implementation tailored for dilated convolution, which effectively accelerates dilated convolution tasks across various configurations. TC-DWC employs a two-step tile reorganization scheme that reconciles the incompatibility between Winograd convolution and dilated convolution. Moreover, it leverages specialized high-throughput tensor cores (TCs), replacing conventional vector units to accelerate the computationally dominant matrix multiplications. To alleviate TC-DWC’s substantial memory footprint, we further develop a multi-stage kernel fusion strategy that eliminates intermediate array allocations, yielding Fused TC-DWC. Experimental results demonstrate that, for single convolutional layers, TC-DWC and Fused TC-DWC deliver average speedups of $1.56\times $ $1.56 \times$ (up to $2.45\times $ $2.45 \times$ ) and $1.29\times $ $1.29 \times$ (up to $1.80\times $ $1.80 \times$ ) over cuDNN optimal implementation, respectively, with Fused TC-DWC reducing memory footprint to 35% of TC-DWC and 55% of cuDNN. For end-to-end inference of CSRNet and DLinkNet34, TC-DWC and Fused TC-DWC deliver average speedups of $1.38\times $ $1.38 \times$ (up to $1.41\times $ $1.41 \times$ ) and $1.20\times $ $1.20 \times$ (up to $1.22\times $ $1.22 \times$ ) over PyTorch baseline, respectively, with Fused TC-DWC enabling support for larger batch sizes.