Anomaly detection remains a prominent challenge in the field of data science, given its extensive range of applications and the array of methodologies at hand. Regrettably, numerous existing anomaly detection techniques exhibit shortcomings, such as ineffectiveness, non-intuitive behavior, or specialization for specific data types. In this research, we adopt a unique fusion of three distinct algorithms to effectively address the challenge of anomaly detection. Initially, we harness the power of autoencoders to reduce the dimensionality of the data, enhancing its manageability while preserving essential patterns. Subsequently, leveraging advanced clustering techniques, we partition our data into K distinct subsets, a strategic approach aimed at pinpointing local anomalies. Lastly, we employ the robust Isolation Forest algorithm to comprehensively identify remaining outliers in the dataset. This orchestrated fusion of methodologies promises to offer a comprehensive and efficient solution for detecting both local and global anomalies, Thus contributing to the development of anomaly detection methods in data science.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Outlier Detection: A Hybrid Architecture with Autoencoder Clustering and Isolation Forest

  • Sanae Borrohou,
  • Rachida Fissoune,
  • Nadia Kabachi

摘要

Anomaly detection remains a prominent challenge in the field of data science, given its extensive range of applications and the array of methodologies at hand. Regrettably, numerous existing anomaly detection techniques exhibit shortcomings, such as ineffectiveness, non-intuitive behavior, or specialization for specific data types. In this research, we adopt a unique fusion of three distinct algorithms to effectively address the challenge of anomaly detection. Initially, we harness the power of autoencoders to reduce the dimensionality of the data, enhancing its manageability while preserving essential patterns. Subsequently, leveraging advanced clustering techniques, we partition our data into K distinct subsets, a strategic approach aimed at pinpointing local anomalies. Lastly, we employ the robust Isolation Forest algorithm to comprehensively identify remaining outliers in the dataset. This orchestrated fusion of methodologies promises to offer a comprehensive and efficient solution for detecting both local and global anomalies, Thus contributing to the development of anomaly detection methods in data science.