Automated recognition of sign languages is critical for improving communication for the deaf and mute community. While Bengali Sign Language (BdSL) has been explored at the alphabet level, word-level recognition remains underdeveloped due to limited datasets and evaluation. In this paper, we conduct a benchmark analysis of three deep learning architectures as 3D CNN, Long Short-Term Memory (LSTM), and Temporal Shift Module (TSM)—on a structured Bengali Word-Level Sign Language (BdWLSL) dataset. These models are compared using accuracy, precision, recall, specificity, F1-score, and mean Average Precision (mAP). We further introduce Explainable AI (XAI) visualizations to better understand each model’s decision-making process. Our findings reveal that while LSTM performs slightly better in handling temporal dynamics, the 3D CNN offers a balanced trade-off in capturing spatio-temporal features. This study aims to serve as a foundational benchmark for future BdWLSL recognition tasks.