A resource-efficient partial 3D convolution for gesture recognition

3DCNNs have shown impressive capabilities in extracting spatiotemporal features from videos. However, in practical applications, the numerous trainable parameters in most 3DCNN models result in longer latency times. Many models attempt to improve computational speed by reducing the number of floating-point operations. However, this approach alone may not effectively reduce latency times. Therefore, this paper proposes partial 3D convolution by extracting features from only a portion of the channels to reduce memory access and latency times. Additionally, structural reparameterization was applied to simplify the inference structure of the partial convolution. This convolution can easily substitute regular convolutions and depthwise convolutions in existing models. Through verification on three datasets, Jester, EgoGesture, and NvGesture, the proposed partial 3D convolution demonstrates the following highlights: (i) low memory access, (ii) significantly lower latency compared to other models, (iii) almost unchanged accuracy. For example, when the regular convolutions of ResNeXt101 and ResNeXt50 are replaced with the proposed partial convolutions on a GPU, runtime latency is reduced by 29.6 and 29.9% respectively, with almost no change in accuracy. Furthermore, computational complexity is also reduced.