Few-shot image generation aims to generate realistic and diverse images for an unseen category with only a few examples. Recent works normally decompose the features of the input images and use random interpolations to reconstruct a new image. However, the primary limitation of this approach is that the feature space is not fully utilized for each generation task. That is, operating directly on the input features will result in a limited diversity for the generated results. To overcome this issue, we propose a novel approach named the Local distribution Estimation and Sampling-based Generative Adversarial Network (Lies-GAN) to dynamically estimate and sample from the task-aware local feature distribution for few-shot image generation tasks. We experimentally demonstrate that the task-level statistic of local features is essential for various few-shot generation tasks, which has not been adequately studied before. Based on this observation, we use a multivariate Gaussian model to estimate a task-aware local distribution for each task. Moreover, instead of directly operating on the input features as in prior works, we generate new local features by randomly sampling from the estimated task-aware local distribution to produce new images. Comprehensive experiments show that Lies-GAN can generate more realistic and diverse images than existing methods.