Detecting Windows Malware Using Machine Learning
摘要
As malware threats continue to evolve in scale and sophistication, the need for automated and behavior-based detection methods has become increasingly critical. In this study, we investigate the effectiveness of various machine learning and deep learning models for malware classification based on API call sequences. Our evaluation encompasses six representative approaches: XGBoost, LSTM, BiLSTM, CNN, CNN + BiLSTM, and BiLSTM with Word2Vec embeddings. We transform API traces into structured and sequential data formats, enabling both bag-of-words and sequence-aware architectures. Comprehensive experiments conducted on a benchmark PE malware dataset reveal that deep learning models, particularly BiLSTM and CNN-based architectures, outperform traditional classifiers in capturing semantic behavior patterns. While XGBoost performs competitively, Word2Vec-based models demonstrate underwhelming results, indicating a need for better pretraining or fine-tuning strategies. Our findings highlight the trade-offs between model complexity, feature representation, and classification accuracy in real-world malware detection scenarios.