<p>Outlier detection has important applications in various fields, e.g., financial fraud detection, network intrusion detection, machine fault diagnosis, etc. However, most existing outlier detection methods focus on datasets with a single attribute type, and fail to effectively handle mixed datasets (i.e., datasets containing both numerical and categorical attributes). Moreover, these methods assume that all attributes have equal weight, and neglect the differences between the influences of different attributes on the detection results. To address these two limitations, this article focuses on the detection of outliers in mixed datasets from the perspective of neighborhood rough sets (NRS) and weighted distance. First, we introduce a new measure called granular neighborhood discrimination index (GNDI) in the NRS framework, and use it to compute the significance and weight of each attribute. Second, we present an outlier detection algorithm (called GNDIWD) based on GNDI and weighted distance. In GNDIWD, we compute the weighted distance on numerical attributes and that on categorical attributes, respectively, and detect outliers by combining the distance outlier factors generated from these two types of distances. Experiments performed on 15 public datasets indicate that GNDIWD outperforms the baseline methods. In addition, the results of statistical tests also demonstrate the effectiveness of GNDIWD. The code is available at <a href="https://github.com/outlier-bot/GNDIWD.">https://github.com/outlier-bot/GNDIWD.</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Outlier detection in mixed data: a method based on granular neighborhood discrimination index and weighted distance

  • Yixuan Jiang,
  • Chuanyu Huang,
  • Feng Jiang,
  • Xun Duan,
  • Yifan He

摘要

Outlier detection has important applications in various fields, e.g., financial fraud detection, network intrusion detection, machine fault diagnosis, etc. However, most existing outlier detection methods focus on datasets with a single attribute type, and fail to effectively handle mixed datasets (i.e., datasets containing both numerical and categorical attributes). Moreover, these methods assume that all attributes have equal weight, and neglect the differences between the influences of different attributes on the detection results. To address these two limitations, this article focuses on the detection of outliers in mixed datasets from the perspective of neighborhood rough sets (NRS) and weighted distance. First, we introduce a new measure called granular neighborhood discrimination index (GNDI) in the NRS framework, and use it to compute the significance and weight of each attribute. Second, we present an outlier detection algorithm (called GNDIWD) based on GNDI and weighted distance. In GNDIWD, we compute the weighted distance on numerical attributes and that on categorical attributes, respectively, and detect outliers by combining the distance outlier factors generated from these two types of distances. Experiments performed on 15 public datasets indicate that GNDIWD outperforms the baseline methods. In addition, the results of statistical tests also demonstrate the effectiveness of GNDIWD. The code is available at https://github.com/outlier-bot/GNDIWD.