<p>People who use sign languages to communicate can express themselves through body language, facial expressions, and hand gestures. People who have hearing loss mostly use it for interpersonal communication. Arab Sign Language (ASL) improves accessibility and communication for the deaf and hard-of-hearing communities throughout the Arab world through a variety of practical uses such as education, healthcare, and media and entertainment. The significance of ASL in promoting social inclusion, education, and communication for the deaf people in Arab nations is demonstrated by these uses. Convolutional Neural Network (CNN) techniques and transfer learning are now the main methods used for the recognition of static sign language in terms of classifying hand sign images. Although many researchers are working on this subject, not many have investigated how Vision Transformers (ViT) might be used to solve the sign language recognition difficulty, specifically with Arabic songs and videos. Even though sign language recognition systems have advanced significantly, there is still a key gap in the literature about using the Vision Transformer (ViT) paradigm to recognize Arabic Sign Language letters from video data. In addition to pointing out a significant research gap, this absence offers a rare chance to use cutting-edge deep learning methods to improve Arabic Sign Language recognition, which will ultimately increase the deaf community’s accessibility to communication. In comparison to previous techniques, Vision Transformers, a recent development in the field of deep neural networks, have demonstrated promising performance with fewer processing resources needed. In this paper, a novel model recognizing the Arab sign language from video with a novel video dataset is proposed. The proposed model is divided into several steps. The first step in the proposed model is preprocessing the dataset with data augmentation. The second step is a modified fine-tuning model that is a video mask autoencoder after removing the decoder phase. The main steps from VideoMAE are the feature extractor with ViT, cube extraction to obtain video tokens, masking with a ratio of 90% to decrease the number of input tokens, and the encoder. The last step is classifying the 12 different classes. The proposed model achieves an accuracy of 99.9%. The high accuracy proves that the proposed model can effectively recognize Arab sign language from video.</p>