Enhancing Explainable Herbal Recognition with Vision Transformer Features and SVM

This paper presents a novel method for automatic plant image recognition by leveraging features extracted from a Vision Transformer (ViT) model and classifying them using a Support Vector Machine (SVM). This approach utilizes the self-attention mechanism of ViT, which captures global relationships within the plant images. Features extracted from ViT are subsequently classified efficiently using SVM. Additionally, we use attention maps to interpret the model’s behavior. Furthermore, this study contributes a new dataset of medicinal plants collected in Vietnam and conducts experiments across five different plant image datasets. Experimental results demonstrate that the ViT and SVM combination outperforms traditional deep learning models and other basic classifiers. The attention maps effectively explain how the model focuses on crucial features in the images, thereby providing high-quality features for the SVM classifier.