Biomedical electronics has become an integral component in the advancement of science, technology, and human welfare. The early detection of cardiovascular diseases through electrocardiogram (ECG) signals using compact devices, as an alternative to expensive specialized equipment, represents an emerging trend. However, implementing artificial intelligence and deep learning algorithms on hardware-constrained devices presents significant challenges. In this paper, we propose an approach for classifying cardiac arrhythmias based on a deep learning model of a Convolutional Neural Network (CNN), incorporating techniques such as hyperparameter optimization, model compression via knowledge distillation, and model quantization. Our results demonstrate that the proposed model achieves an overall accuracy of 99.6% while maintaining a small footprint of approximately 0.08 MB. Furthermore, the processing time on embedded devices averages 8.7 ms per sample.