During neural machine translation (NMT) tasks, we observe that despite the common assumption that increasing encoder depth leads to improved performance, this effect is less pronounced in low-resource scenarios and can even exacerbate overfitting issues. Our comparative analysis between NMT models equipped with shallow and deep encoders reveals that the majority of sentences are more effectively translated by a shallow encoder. Further analysis indicates that these sentences tend to be simpler, suggesting the shallow encoder’s ability to capture unique features in simple text. Building on these insights, we introduce NATASHA, a novel training strategy that enhances the capabilities of deep models in low-resource Neural mAchine TrAnslation with SHallow feAtures extracted through sequence-level knowledge distillation from the shallow model. Experimental results on five low-resource NMT tasks show that NATASHA consistently improves over strong baselines by at least 1 BLEU point. Furthermore, when combined with other regularization methods, NATASHA achieves leading-edge performance on the IWSLT14 De-En translation task. Further analysis of our method’s effectiveness reveals that integrating shallow features reduces the complexity of the training data, facilitating deep models in learning patterns and features within simple text during the early stages of training. This unleashes the deep model’s ability to learn representations of low-frequency words and long sentences, thereby enhancing overall performance.