Sector-Wise Backpropagation for Low-Resource Text Classification in Deep Models
摘要
This work presents a modular training approach for deep neural networks applied to text classification, introducing the concept of a sector as a trainable layer and its subsequent non-trainable layers up to the next trainable layer. Each sector is trained independently using auxiliary models that mimic the original network’s output layer, allowing for training without full end-to-end backpropagation. The method was evaluated on 1D ConvNet, Transformer, and bidirectional LSTM architectures across five benchmark text classification data sets, using 1%, 5%, and 10% of the data for training. Results show that sector-wise modular training achieves comparable or even superior accuracy to traditional end-to-end training while reducing computational time, particularly in four out of five data sets. This approach offers a generalizable alternative to standard training methods in deep learning for text classification.