<p>Traditional speech forgery detection methods rely on handcrafted features, which are based on fixed computational formulas. The advantages of them are computational efficiency and operational stability, performing well particularly in resource-constrained environments. However, they face issues such as inadequate adaptability and weak generalization ability when evaluated on diverse datasets. To address these problems, our paper proposes a new spectrum feature reconstruction framework. This framework uses VQVAE as the generator to reconstruct speech spectrograms, and then fuses the reconstructed and original spectrogram to create frontend features. This approach amplifies the discriminative features related to speech forgery detection while preserving feature stability. To further refine our framework, we propose a Knowledge-Driven Generation Optimization training strategy. Specifically, we use the target classifier’s Encoder to supervise the generator, ensuring that the reconstruction is aligned with the target classifier’s objectives. Besides, we propose the Dual-Margin triplet loss by applying a positive sample constraint and introduce the Genuine Speech Consistency Loss to further guide the generator. Extensive experiments show that the detector based on this framework significantly improves detection accuracy and generalization compared to using original handcrafted features, both on in-distribution and out-of-distribution dataset.</p>