BERT-Enhanced Dual-Attention RNN for Short Text Spam Detection
摘要
Detecting spam is a crucial challenge for a user's safety and privacy. Unlike long texts such as emails, short text messages have a limitation in the number of words, making it more challenging to analyze. Artificial intelligence techniques and Machine learning approaches for spam detection often face challenges due to modern spam’s dynamic and context-specific nature. To solve this issue, we proposed a novel method consisting of these key steps: pre-processing, word embedding using BERT, self-attention for feature weighting, RNN for temporal feature extraction, temporal attention for feature selection, and a fully connected layer for classification. The model generates rich word representations by utilizing BERT, while its dual-attention mechanism enables it to concentrate on significant words and patterns within the message. The RNN establishes the relation between the words in the sentences. We evaluated the proposed model on the most famous dataset (UCI SMS Spam v.1). The proposed method achieved an accuracy of 98.9%, surpassing the state-of-the-art techniques.