Latent Diffusion Models (LDMs) have shown great potential for generating high-quality images. However, applying LDMs to medical imaging, especially MRI reconstruction, tends to generate blurry and low-fidelity images, especially for highly accelerated MRI. To address these difficulties, in this work, we propose a novel detail-preserving LDM, which shows great reconstruction performance for brain MRI with 16 \(\times \) acceleration. Specifically, we propose to perform pixel-wise correction based on data fidelity constraints on conditional LDM to preserve fine-grained data fidelity. Besides, to alleviate the influence of the pre-trained highly nonlinear decoder, we introduce an additional correction term on the latent representation for adjusting autoencoder self-regression error. Experiments on the public fastMRI dataset show that our model outperforms the existing SOTA methods significantly, especially in regions containing fine-grain structures with high acceleration factors. Our code is available at the GitHub repository .