A Novel Dynamic Programming Method for Non-parametric Data Discretization

Discretization is a data pre-processing technique that transforms continuous data attribute values into a finite set of intervals. In the realm of supervised discretization approaches, the process is treated as an optimization problem, necessitating the development of an objective function based on various criteria, including entropy, mutual information, class-attribute interdependency, and others. Previous research in this direction typically involved user-defined parameters and heavily relied on either greedy or heuristic strategies to explore continuous variables’ range space, potentially leading to local optima or failing to achieve global optimization. This paper introduces a non-parametric approach for univariate and supervised data discretization utilizing a dynamic programming algorithm. We provide theoretical proof that our proposed algorithm can achieve a global and precise solution for optimal split of a continuous variable. To evaluate the performance of our approach, we conducted experiments on publicly available benchmark datasets to showcase its efficiency and robustness. Empirical results demonstrate that our method outperforms existing state-of-the-art approaches in terms of attaining the optimal correlation value, particularly when applied to large-scale datasets.