Modern applications must ensure network services; some conventional applications could function well without a service level requirement. Managing the scalability for new service designs, setting up control protocols for the purpose of routing, along with allocating data for the designated traffic streams are just a few of the challenges that must be addressed to alleviate the mounting load on computer networks’ performance. This study addresses the challenge of accurately classifying low-frequency traffic data, which leads to class imbalance and misclassification in traditional machine learning models. We propose a cost-sensitive convolutional neural network (CSCNN) model trained on the ISCX VPN-nonVPN dataset, using a cost matrix to penalize misclassification based on class distribution. Experimental evaluation shows that CSCNN outperforms other models like Deep CNN, DFR CNN, and SMOTE + CNN, achieving the highest precision of 99.85% and significant gains in recall for minority classes. It has also proven to be more effective in data classification compared to other methods. Furthermore, the SMOTE + CNN approach improved recall by 2.5% over Deep CNN and 1.8% over DFR CNN, demonstrating its effectiveness in handling class imbalance while maintaining high classification performance.