Physical violence in schools represents a global problem with immediate impact and long-term effects on students’ academic performance and well-being. This work in progress aims to detect physical violence in elementary schools using computer vision and machine learning. In order to perform the detection task, first, an image dataset was constructed from open-access repositories for training, validation, and testing. Then, local binary patterns and principal component analysis were used to extract relevant features of the images and reduce the dimensionality of feature vectors for the decision trees model; convolutional neural networks and vision transformers extracted and reduced features automatically. After that, three classification algorithms were tested: convolutional neural networks, decision trees, and vision transformers. Preliminary experimental results show that the vision transformer model produces the best results with over 91% accuracy and F-measure.