Though deep learning models exhibit good performance, it’s usually challenging to deploy them in low-resource devices and embedded platforms for human-to-machine interaction that are often faced with challenges of limited computational resources. In this paper, we carried out comparative experiments to ascertain the robustness of binarized neural networks (BNNs) and deep learning models that use knowledge distillation in relation to their performance on the University of Glasgow radar data for activity recognition. We evaluated the performance of three ImageNet models and used the best deep learning model among them as a teacher to aid learning for a low-parameterized model (student) in a knowledge distillation paradigm. We also evaluated the performance of BNNs on the same dataset. While both approaches exhibit comparable performance, the student model size and loss values are far smaller than the BNN model. However, the BNN model exhibits a better performance in terms of accuracy and confusion ratio which makes the choice of these deep learning approaches for low-resource devices a trade-off between accuracy and model size.