Molecular optimization represents a primary challenge in drug discovery, aiming to simultaneously optimize multiple chemical properties with high molecular similarity. Traditional methods have faced challenges including overfitting, low optimization efficiency and diversity. To address these challenges, we propose a latent diffusion model named LDMO for molecular optimization that achieves precise modeling of data distributions while balancing diversity and quality. LDMO selectively incorporates Gaussian noise to preserve conditional features as guidance information, generating molecules with specific properties under structural constraints. With regard to the issue that the discreteness of molecular entities results in low efficiency and poor controllability, we offer a way that leverages the latent space of a pretrained autoencoder to learn a high-quality diffusion model for molecular optimization. Furthermore, we design a contrastive process that minimizes noise-induced impacts on molecular properties. Experimental results demonstrate that LDMO generates novel molecules with enhanced chemical properties and high structural similarity to original molecules.