错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Clustering Based Undersampling for Effective Learning from Imbalanced Data: An Iterative Approach

  • Rajdeep Bhattacharya,
  • Rajonya De,
  • Anuran Chakraborty,
  • Ram Sarkar

摘要

The class imbalance problem is prevalent in many classification tasks such as disease identification using microarray data, network intrusion detection, and so on. These are tasks in which the class distribution is skewed towards one class, more commonly known as the majority class. In such cases, traditional classifiers may not perform well as they tend to become biased towards the majority class. To address this problem, an intelligent undersampling technique is proposed in this paper. The method first groups the samples of the majority class into \(l\) l clusters, where \(l\) l is some number, using the K-means clustering algorithm. From these clusters, each of the cluster centroids is selected to form the undersampled majority class set. A classifier is then trained on this undersampled dataset consisting of the selected majority class samples and all the minority class samples. The trained model is used to predict the probability of each majority class sample belonging to the minority class. A Gaussian distribution is then constructed from these probabilities using which the top p-percent samples from each cluster are selected. The centroid of the cluster is recomputed using these samples only, which forms the new sample for our dataset for the corresponding cluster. The classifier is again trained using these samples, along with the minority class samples, thereby iteratively improving the classifier. The results obtained by the proposed method show that it performs better than most state-of-the-art methods while being evaluated on some standard datasets.