MFL-RAT: Multi-class Few-Shot Learning Method for Encrypted RAT Traffic Detection

Remote Access Trojan (RAT) poses a significant risk in today’s network environment. These malware are commonly employed by hacking groups to monitor victims’ activities and illicitly obtain sensitive personal information from targeted computers. With the rapid upgrade of RAT, obtaining sufficient network traffic samples of newly emerging RATs is difficult in a short period. Unfortunately, the existing malicious traffic detection methods typically rely on a large volume of traffic samples for effective learning, resulting in a limited ability to recognize newly emerging RATs effectively. Thus, we propose a novel multi-class encrypted RAT traffic detection method called “MFL-RAT” based on the meta-learning model Model-Agnostic Meta-Learning (MAML). In addition to distinguishing RAT traffic from benign traffic as before, our approach further considers classifying traffic from different new RAT classes, which is more practical in real-world scenarios. MFL-RAT leverages prior knowledge, generalizing from original RATs with sufficient network traffic samples, and develops a robust generalization model. It then quickly fine-tunes the model through a handful of new RAT traffic samples, and the fine-tuned model enjoys a high detection accuracy for new RATs. MFL-RAT employs a multimodal feature extraction network that generates flow sequence and packet payload embeddings. By combining these embeddings through feature fusion, MFL-RAT maximizes the utilization of information from limited resources. MFL-RAT performs better than existing works with at least a precision of 96.8 \(\%\) and a recall rate of 96.1 \(\%\) when dealing with different new RATs.