<p>Nowadays, hand gestures are emerging as a popular mode of interaction with digital technologies and intelligent systems. While existing input methods, such as touch, buttons, and voice interfaces, are widely used, they do not adequately support all users. In particular, visually impaired people (VIP), who cannot effectively access visual content, face significant challenges. For them, hand gestures provide a crucial means of enabling natural and accessible human–machine interactions. In this context, this work presents a robust deep learning framework for hand gesture recognition in VIP, leveraging a hybrid architecture that integrates a Convolutional neural network (CNN) with Transformer encoder. The proposed architecture achieved 99.44% test accuracy across 15 distinct gesture classes on a custom-designed dataset, demonstrating strong generalization and robustness. The dataset consists of 19,614 grayscale images from participants, with 13 males and 7 females. The CNN effectively extracted hierarchical spatial features from the input images. Subsequently, the Encoder captures global contextual dependencies, improving the model’s ability to distinguish between gesture variations. In addition, an ablation study has been conducted with basic CNN, VGG16, and EfficientNetB4 to assess the generalizability and adaptability of the proposed dataset. Moreover, the proposed model has achieved the fastest inference time, nearly 0.4 seconds, and the highest accuracy under the 5-fold strategy against VGG16. The overall results of this work demonstrate the effectiveness of the proposed model across 15 classes, among the baseline CNN and even the advanced pre-trained models. Ultimately, the proposed framework achieved 99.44% accuracy and an inference time of 445 ms, demonstrating its effectiveness for robust, real-world human-machine interaction applications.</p>