Recent advances in class-incremental learning leverage large pre-trained models (PTMs) for powerful feature extraction. Nonetheless, existing methods still suffer from two key challenges: (i) parameter drift, where fine-tuning weights on new classes progressively erodes previously established decision boundaries (catastrophic forgetting), and (ii) feature overlap, where newly introduced class embeddings collide with existing clusters in the frozen PTM feature space. We propose a method of knowledge modeling and distribution refinement (KMDR) to address both issues simultaneously. KMDR encodes each class distribution as a compact Gaussian mixture model, which serves as a fixed-size parametric memory and eliminates the need to store raw exemplars. It further trains a single lightweight projector with an intra-class variance-suppression loss to tighten clusters. The proposed method requires neither task-specific modules nor rehearsal buffers, yet it mitigates parameter drift and feature overlap throughout long task sequences. Extensive evaluations across CIFAR-100, CUB-200, ImageNet-A, and ImageNet-R benchmark datasets demonstrate that KMDR consistently outperforms state-of-the-art baselines.