The contribution of this paper is to provide an entire framework for anomaly detection on the USPS image dataset by means of the combination of the Typicality and Eccentricity Data Analytics (TEDA) and PySpark-based streaming and MLAutoCloud benchmarking system. The primary objective is to assess the performance of TEDA in a big data streaming context and to compare its efficiency with those of automated machine learning pipelines. The PySpark implementation permits scalable processing of streaming image data and emulating an online anomaly detection pipeline with the Mahalanobis distance and TEDA. Meanwhile, the dataset is tested by batch learning under the inspiration of MLAutoCloud, in terms of accuracy, interpretability and latency. Large amount of visualization using PCA projections, Mahalanobis score distributions, and KMeans clustering confirms the detection of atypical samples. Our experimental results show that TEDA-PySpark pipeline has better interpretability and efficiency on streaming settings, while MLAutoCloud achieves better classification accuracy over static data. This hybrid experimentation also provides new perspectives how to place explainable anomaly detection systems to distributed architectures for practical computer vision applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TEDA-Powered Anomaly Detection on USPS Images Using PySpark Streaming and MLAutoCloud Framework: A Comparative Study

  • Madhuri Parekh,
  • Madhu Shukla

摘要

The contribution of this paper is to provide an entire framework for anomaly detection on the USPS image dataset by means of the combination of the Typicality and Eccentricity Data Analytics (TEDA) and PySpark-based streaming and MLAutoCloud benchmarking system. The primary objective is to assess the performance of TEDA in a big data streaming context and to compare its efficiency with those of automated machine learning pipelines. The PySpark implementation permits scalable processing of streaming image data and emulating an online anomaly detection pipeline with the Mahalanobis distance and TEDA. Meanwhile, the dataset is tested by batch learning under the inspiration of MLAutoCloud, in terms of accuracy, interpretability and latency. Large amount of visualization using PCA projections, Mahalanobis score distributions, and KMeans clustering confirms the detection of atypical samples. Our experimental results show that TEDA-PySpark pipeline has better interpretability and efficiency on streaming settings, while MLAutoCloud achieves better classification accuracy over static data. This hybrid experimentation also provides new perspectives how to place explainable anomaly detection systems to distributed architectures for practical computer vision applications.