Rethinking the temporal downsampling paradigm for continuous sign language recognition
摘要
Pooling layers are necessites in current deep convolutional neural networks (CNNs) for increasing receptive fields and producing robust downsized representations. However, for temporal-related tasks, how to better preserve discriminative features in the pooling process is rarely visited, while directly employing commonly-used pooling methods may not be optimal. In this paper, we propose local discriminative pooling (LDP), a simple, general yet effective tool to dynamically aggregate features in the sampling window based on their discriminativeness for robust downsized temporal representations. Compared with other spatial and temporal pooling methods, extensive experiments on continuous sign language recognition (CSLR) verify the superiority of LDP. Moreover, experimental results upon multiple backbones on two large-scale CSLR datasets show the generalizability of LDP. Comparison with other state-of-the-art methods verify the effectiveness of our proposed method. Code is released at https://github.com/CaifengLiu/LDP.