This research introduces a lightweight neural audio enhancement system to re-fine speech signals processed by upstream models, such as MetricGAN +, mitigating residual distortions while retaining noise suppression benefits. The system employs a 192-dimensional (192D) speaker embedding from original clean audio and a 10-point quantized pitch contour to restore speaker identity and prosodic naturalness. Objective evaluations show significant improvements in speaker embedding cosine similarity (SECS) from 0.79 to 0.83 (p < 0.05) and perceptual evaluation of speech quality (PESQ) from 1.75 to 1.91 (p < 0.05) compared to the baseline. Subjective evaluations, conducted under controlled conditions with 20 listeners, yield a mean opinion score (MOS) of 4.2 (SD = 0.3), surpassing the baseline’s 4.0 (SD = 0.3). Listeners reported enhanced audio clarity due to effective noise and artifact removal, with speaker identity preserved so closely that differences are only detectable upon meticulous inspection. These attributes make the system ideal for real-time applications like teleconferencing and voice assistants.