Agricultural husks, such as those from rice, wheat, and legumes, are valuable agro-waste materials widely used in energy production, biodegradable packaging, and livestock feed. Efficient identification and classification of these husks are essential for optimizing their reuse and supporting sustainable agricultural ecosystems. This study presents a hybrid deep learning model integrating Vision Transformer and DenseNet121 architectures to classify agricultural husks effectively. Utilizing the BDHusk dataset, which contains 16,800 high-quality images across eight distinct husk classes collected from Bangladesh, the proposed method achieved remarkable classification performance, with training, validation, and test accuracies of 98.88%, 98.85%, and 98.81%, respectively. The robustness of the model was further validated by impressive scores across multiple metrics, including a Cohen’s Kappa of 0.9864, a Matthews Correlation Coefficient (MCC) of 0.9864, and a Fowlkes-Mallows Index of 0.9764. Moreover, the model demonstrated superior discriminative capability, achieving Area Under the Curve (AUC) scores exceeding 0.999 across all classes. To enhance transparency and interpretability, explainable AI techniques such as LIME, Grad-CAM, and Grad-CAM++ were employed, highlighting critical image regions influential in decision-making processes. This research significantly contributes to automating husk classification, thereby reducing labor costs, enhancing sorting efficiency, and promoting sustainable agricultural practices in alignment with circular economy principles.