Detecting human abnormal actions, such as falling, can reduce the cost for both economic and health systems, as well as protect the individuals involved. However, training an anomaly detector from scratch can be computationally expensive. We propose reusing the representations learned by lightweight Graph Neural Networks (GNNs) from a multi-action classification task for quick and accurate anomaly detection. Specifically, the representations are extracted by a lightweight GNNs obtained through self-distillation and are then used for unsupervised anomaly detection with effective machine learning methods such as Gaussian Mixture Model, Dirichlet Process Mixture Model, Isolation Forest, and Local Outlier Factor. This hybrid approach is evaluated on the NTU RGB+D and a subset of Kinetics400 datasets across 20 anomaly detection tasks, including speed-sensitive tasks, exercise actions, and daily activities. The feature extractors used are ST-GCN and AAGCN. Our method achieves an average AUC improvement of \(11.4\%\) compared to the unsupervised top model GEPC across the anomaly detection tasks on NTU RGB+D dataset. It also outperforms another supervised method on the URFall dataset. The representations obtained through self-distillation were superior in 8 out of 10 anomaly detection tasks on NTU RGB+D. Additionally, the lightest AAGCN model, which is around \(60\%\) lighter than the heaviest model, shows similar or superior performance on average across all anomaly detection tasks for both datasets. However, the representations extracted are usually redundant for anomaly detection, providing more information than anomaly detection needs because their extractors were trained on multi-action classification tasks. Consequently, we also did experiments across all tasks to show that feature reduction improves even more detection performance by up to \(17.69\%\) on the subset of Kinetics400 and up to \(24.45\%\) on NTU RGB+D.