Cardiovascular diseases (CVDs) remain the leading cause of mortality globally, and early detection of abnormal heart rhythms is crucial for improving treatment outcomes. Electrocardiogram (ECG) signals are essential for identifying these abnormalities. Recently, Machine Learning (ML), especially Deep Learning (DL) methods have been successfully applied to ECG classification. However, the variation in datasets and preprocessing techniques in existing studies complicates their real-world clinical applications. This paper provides an overview of ECG classification and focuses on two major heartbeat abnormalities—supraventricular ectopic beats (SVEB) and ventricular ectopic beats (VEB)—and two critical ECG classification tasks—atrial fibrillation (AFIB) and ST-segment and T-wave changes (STTC). We evaluate these abnormalities using raw data selected from 18 public ECG datasets, processed with unified preprocessing methods, and tested with 13 classification models. The compiled datasets and codes are available at https://github.com/ZhangCWei/ECG_Benchmark