Few-shot font generation aims to generate all characters of a certain font by using its very few seen characters as references. Recent studies assumed that a target font can be regarded as a mixture of several source fonts, namely basis fonts, and the style of the target font can be generated by combining several latent representations respectively captured from a group of other fonts with given weights. However, these basis fonts along with their corresponding mixing weights are unlearnable as they are determined by a pre-trained network for font feature extraction. The styles in their basis fonts also have significant differences so it is not flexible to fusion them. In this paper, we present an adaptive basis fonts and weights learning (ABW) module to learn more appropriate basis fonts and weights, making the results more similar to the target. We project the reference characters into a Gaussian Mixture Model (GMM) distributed latent space, where each latent Gaussian collects latent representations of fonts with similar styles. An Rival Penalized Competitive Learning (RPCL) enhanced Expectation Maximization (EM) like learning algorithm is introduced to learn the GMM-structured latent space, jointly with the training of the network. The ABW module is also able to automatically determine an appropriate number of Gaussians in GMM, i.e., the number of basis fonts, making font mixing more flexible. We collect a dataset of 500 Chinese fonts with 6.5k characters each and evaluate our method on it. Experimental results demonstrate that our method outperforms recent few-shot font generation methods.