Scholarlink

in this paper, methods for evaluating the quality of a new dataset using statistical analysis and classical machine learning are presented. The proposed machine learning approach is based on comparing a new dataset with a known reference sample of high quality. This process is carried out using a supervised machine learning algorithm and solving a classification problem. As a result fitted algorithm, it is necessary to estimate its quality of the ROC-AUC \(^{1}\) (Receiver Operating Characteristic - Area Under Curve) metric, on the basis of which a conclusion is made about the quality of the new sample. As an additional measure of comparison, the PSI \(^{2}\) (Population Stability Index) is taken into consideration as a statistical indicator that provides an approximate qualitative evaluation of the difference for 2 distribution densities.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Data Quality Estimation Using Machine Learning Approach and Statistical Metric

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Daniil D. Devyatkin

摘要

in this paper, methods for evaluating the quality of a new dataset using statistical analysis and classical machine learning are presented. The proposed machine learning approach is based on comparing a new dataset with a known reference sample of high quality. This process is carried out using a supervised machine learning algorithm and solving a classification problem. As a result fitted algorithm, it is necessary to estimate its quality of the ROC-AUC \(^{1}\) (Receiver Operating Characteristic - Area Under Curve) metric, on the basis of which a conclusion is made about the quality of the new sample. As an additional measure of comparison, the PSI \(^{2}\) (Population Stability Index) is taken into consideration as a statistical indicator that provides an approximate qualitative evaluation of the difference for 2 distribution densities.