Multi-modality imaging plays a crucial role in clinical diagnosis. Reconstructing missing modality images, such as CT-to-MR, is quite important when only one modality is available. Previous works either fall short in preserving the anatomical structures during translation or require paired data, leaving significant challenges unaddressed in the realm of unpaired medical image translation. This study introduces a novel structure-preserving diffusion model specifically designed for unpaired medical image translation, leveraging edge information to represent common anatomical structures across different modalities. To bridge the domain gap effectively, we further propose a novel Interleaved Sampling Refinement (ISR) mechanism that dynamically alternates the use of edge information. This approach not only generates high-quality images but also preserves structural integrity across modalities. Our experiments conducted on two public datasets have achieved the state-of-the-art performance, demonstrating the advantage of our method on unpaired medical image translation. The code of our implementation is available at GitHub .