Preserving data distribution in sampling and instance selection with Renyi’s divergence

This paper introduces a novel method for sampling based on a distribution function. By utilizing Renyi’s divergence criterion, a recursive formulation is derived directly from the difference between the original and estimated distributions. This recursive equation is solved using the gradient descent algorithm, from which new samples are then generated. Because this method relies on the original data’s distribution, it effectively preserves the data distribution in the sampled dataset. When the original distribution is unknown, kernel density estimation is used for approximation. Experimental results show that the proposed method successfully maintains the data distribution and concept integrity, while also preserving the model’s predictability on selected instances. Specifically, this method has managed to reduce the dataset size by approximately \(70\%\) $70\%$ , without compromising the accuracy of the learning algorithm.