Top-N recommendation is widely accepted as an effective method in personalized service that well serves users of different interests. However, as analyzed from the SOTAs, their performance on the users with diverse activity levels has significant distinction, which seriously damage the service quality of personalized recommendation. Existing studies do not pay a high attention to this issue, which simply assume the preference of all users follows a common probability distribution and then use a fixed schema (e.g., one latent vector) to model user representation. This assumption makes existing models hard to accommodate users of diverse activity levels. In this work, we propose a Variational Kernel Density Estimation (VKDE) model, a non-parametric estimation, which aims to fit arbitrary preference distributions for users. VKDE constructs user (global) preference distribution with multiple local distributions collectively. We propose a variational kernel function to infer user one-faceted interests and generate each local distribution. A sampling strategy for user one-faceted interest is further proposed to reduce training complexity and keep the recommendation effectiveness. Our experimental results on three public datasets show that VKDE outperforms SOTAs and greatly improves the accuracy for users of diverse activity levels.