Unsupervised Anomaly Detection in Cellular Modem Metrics Using Deep Autoencoders
摘要
This paper addresses the challenge of unsupervised anomaly detection in key metrics related to data transmission in fourth and fifth-generation (4G and 5G) cellular networks, with the main aim of developing a monitoring service that can alert on abnormal behavior in real-time. The dataset used in this research includes channel and upper-layer performance metrics collected from several cellular modems deployed on a remote-controlled ship. Deep autoencoders (DAE) were selected as the main approach due to their ability to detect point, contextual, and collective anomalies. Several unsupervised machine learning algorithms and preprocessing techniques were applied and optimized to overcome the problem of using DAEs for unsupervised anomaly detection, namely their need for training data containing no known anomalies. Different types of deep autoencoders, including classical, variational, denoising, and robust models, as well as different neural network architectures, consisting of linear and recurrent layers, along with the additional loss function terms, such as contractive and sparse regularizations, have been explored. The presented approach is evaluated using anomalies generated by unsupervised algorithms and special samples that introduce noise and shuffling into normal data, thus affecting the data distribution. Finally, a post-analysis is performed by adding contextual information to the data to help interpret the detected anomalies introducing another layer of validation to the presented approach.