Equipment operating in harsh environments relies on accurate non-destructive testing (NDT) to ensure structural safety. Alternating current field measurement (ACFM) is a relatively new NDT technique widely used for surface defect detection and characterization. As a crucial step, defect reconstruction provides visual representations of defect geometry, supporting maintenance and safety evaluation. However, due to the complex and uncertain mapping between defect geometry and detection signals, existing methods often reconstruct only partial features, failing to capture the complete structure. To address this, we propose a defect reconstruction method based on a directional denoising diffusion model (DDDM). The reconstruction task is modeled as a gradient image recovery problem. Magnetic signals around the defect region are collected using ACFM, and a gradient imaging algorithm generates an initial low-quality defect image. The diffusion process is then decoupled into two components: residual diffusion and noise diffusion. Residual diffusion represents a directional process from the actual defect image to the gradient image, guiding the reverse reconstruction; noise diffusion simulates random disturbances during the recovery. Additionally, we modify the reverse generation mechanism and training objective of the diffusion model to better represent the reconstruction process. Experimental results show that the proposed DDDM outperforms existing methods in reconstruction accuracy, achieving mean errors of 0.406 mm in defect length and 0.417° in defect angle.