Effective Encrypted Traffic Analysis

Network traffic analysis involves the analyzing of the captured traffic data to monitor the performance and security, in order to detect the malicious content within the traffic. It has been a problem for security analysts to detect malware in increased encrypted traffic in the network without violating privacy and cost of detection. This paper discusses the methods for detecting malware in encrypted traffic without decryption and with high accuracy by finding new features (flow-based) and machine learning methods. Features are extracted from encrypted pcap files using Bro Intrusion Detection System (IDS), and it extracts the information from packet capture files and store them into protocol-based log files. Support vector machine’s classification techniques are used to detect the malware. Features selection is used for optimization, and particle swarm optimization (PSO) technique is utilized for this experiment considering its advantages of less complexity. SVM and SVM with PSO are used to obtain malware detection results.