Cardiovascular diseases continue to pose significant health challenges globally, and using a stethoscope to listen to heart sounds (auscultation) remains a prevalent method for screening potential problems. Nevertheless, interpreting these sounds—known as phonocardiograms (PCG)—requires expertise and can be subjective, which limits its accessibility in various environments. While automated tools can assist, most only classify recordings as either “Normal” or “Abnormal.”In this study, we introduce a two-stage deep learning framework aimed at advancing heart sound analysis, utilizing cloud infrastructure. During the first stage, we employ a Convolutional Neural Network (CNN) trained on Mel spectrograms to categorize PCG recordings from the PhysioNet Challenge 2016 dataset as Normal or Abnormal. To tackle the class imbalance, present in the data, we implement class weighting, which enhances the model’s ability to detect abnormal instances. When an abnormal recording is identified, the second stage is activated. In this stage, a second CNN (with its architecture explained in the paper) further classifies the abnormal cases into more specific categories. Unlike keyword-based methods, our approach directly maps clinical diagnosis labels (e.g., MVP, AS, MR grouped under ‘Murmur/Valve’; CAD, MPC under ‘Other Abnormal’) from the metadata of the dataset, rendering it more clinically relevant. To facilitate scalability and efficient data management, the entire system is constructed using AWS S3, which stores the audio files, labels, and trained models. Our first-stage model achieved a validation accuracy of 75.9% and an AUC of 0.77, with class weighting significantly enhancing the model’s capability to accurately recognize abnormal instances. Overall, this framework illustrates a practical and scalable method for automated, multi-level analysis of heart sounds, providing more diagnostic insight in remote or resource-limited settings.