The increasing use of encryption protocols such as Transport Layer Security (TLS) has led to enhanced privacy and security for internet users. However, this development has also enabled malware to conceal its communications within encrypted channels. In this work, we explore the application of machine learning techniques to detect malware in encrypted network traffic. To this end, we compare two distinct approaches: one based on statistical flow features and the other one based on TLS fingerprinting (JA4+). In order to accomplish this objective, we have developed and evaluated two state-of-the-art solutions—a MalDIST-inspired model and JA4+ fingerprint-based classification. The experimental results, based on a curated dataset, show that both models exhibit high levels of accuracy. Notably, JA4+ fingerprinting offers a favorable trade-off between accuracy metrics and overall processing speed compared to the MalDIST-inspired model, making it a promising candidate for deployment in real-world environments. Furthermore, we introduce JA4TS, another fingerprinting technique that focuses on TCP SYN-ACK packets, enhancing the capability of the JA4+ framework to predict malware by identifying TCP/IP stack characteristics, a subject of particular relevance as SNI encryption becomes more prevalent. These findings underscore the efficacy of lightweight, metadata-based models for malware detection in encrypted traffic, particularly in the context of IoT and IIoT networks, where privacy and efficiency are paramount.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Detecting Malware in Encrypted Network Traffic Using Machine Learning and TLS Fingerprints

  • Eduardo Polo-Peyres,
  • Jorge E. López de Vergara,
  • Gustavo Sutter,
  • Iván González,
  • Luis de Pedro

摘要

The increasing use of encryption protocols such as Transport Layer Security (TLS) has led to enhanced privacy and security for internet users. However, this development has also enabled malware to conceal its communications within encrypted channels. In this work, we explore the application of machine learning techniques to detect malware in encrypted network traffic. To this end, we compare two distinct approaches: one based on statistical flow features and the other one based on TLS fingerprinting (JA4+). In order to accomplish this objective, we have developed and evaluated two state-of-the-art solutions—a MalDIST-inspired model and JA4+ fingerprint-based classification. The experimental results, based on a curated dataset, show that both models exhibit high levels of accuracy. Notably, JA4+ fingerprinting offers a favorable trade-off between accuracy metrics and overall processing speed compared to the MalDIST-inspired model, making it a promising candidate for deployment in real-world environments. Furthermore, we introduce JA4TS, another fingerprinting technique that focuses on TCP SYN-ACK packets, enhancing the capability of the JA4+ framework to predict malware by identifying TCP/IP stack characteristics, a subject of particular relevance as SNI encryption becomes more prevalent. These findings underscore the efficacy of lightweight, metadata-based models for malware detection in encrypted traffic, particularly in the context of IoT and IIoT networks, where privacy and efficiency are paramount.