Deep Learning-Based Anomaly Detection in TLS Encrypted Traffic

The growing trend of encrypted network traffic is changing the cybersecurity threat scene. Most critical infrastructures and organizations enhance service delivery by embracing digital platforms and applications that use encryption to ensure that data and Information are moved across networks in an encrypted form to improve security. While this protects data confidentiality, cybercriminals also leverage encryption using cryptographic protocols such as SSL/TLS to launch malicious attacks. This hidden threat exists because of the SSL encryption of benign traffic. Hence, there is a need for visibility in encrypted traffic. This study was conducted to detect malware in encrypted network traffic without decryption. The existing solution which involves bulk decryption, analysis, and re-encryption is prone to privacy issues, not cost-efficient, and time-consuming, creating huge overhead on the network. To address the challenges introduced by decryption, we propose an intelligent framework that strikes a balance between security and privacy to detect malicious activities using extracted flow metadata to train three machine learning models: machine-learning model. It further deployed this set of features as input to an autoencoder, leveraging the construction error of the autoencoder for anomaly detection.