Log Analysis for Feature Engineering and Application of a Boosting Algorithm to Detect Insider Threats

The insider threat has captured the attention of a large number of researchers, as a sensitive and critical issue for most organizations in today’s digital world. It is also a major source of information security and can cause more damage and financial loss than any other threat. In this article, we’ve used feature engineering for features that represent users’ day-to-day activities. We tried different machine learning models such as random forest, xgboost and Catboost. Since the data used to detect malicious activity is unbalanced, the target audience is small. We used KMeansSmote to balance the classes of learning so that the algorithms can learn both classes well. And we used the catboost algorithm to identify the malicious user. The dataset used to evaluate this model is Cert v4.2. CatBoost outperformed other models with the highest F1-score of 95%.