Wireless Federated Learning (WFL) leverages distributed computation on mobile devices to train AI models without sharing raw data, thereby offering a wide array of application scenarios. However, practical deployment faces significant challenges due to device-level data heterogeneity, data scarcity, and transmission latency. To address these issues, this paper proposes a generative model-aided WFL framework, where (1) the data augmentation model is utilized to generate data for improving model efficiency and estimating the federated client contributions; (2) a selection algorithm is integrated that optimizes the balance between network resources and client contributions. Meanwhile, Shapley value from cooperative game theory is employed to measure the contribution of a client, and network resources are modeled in terms of delay. Then we select clients by computing the Upper Confidence Bound (UCB) to achieve an optimal trade-off between dynamic resource exploration and historical experience utilization. Experiments demonstrate that our selection strategy consistently outperforms many methods in scenarios characterized by extreme client data heterogeneity and scarcity.