JPEG is a widely adopted image compression standard, but it often introduces severe compression artifacts such as blocking and blurring, especially under low-quality settings. Existing artifact correction methods predominantly operate in the pixel domain, relying on predefined filters or handcrafted feature extraction, which fail to capture high-frequency details and frequently result in over-smoothing. Recent advances in diffusion models show promise for image restoration but are primarily confined to the pixel domain, neglecting the critical frequency domain information. Additionally, traditional diffusion models are often computationally expensive and slow, limiting their practical applicability. To address these limitations, we propose a novel efficient dual-branch diffusion model that operates jointly in the pixel and frequency domains. The pixel branch ensures global consistency and naturalness, while the frequency branch recovers high-frequency details and mitigates the artifacts introduced during JPEG compression. Our approach leverages wavelet transform to effectively integrate frequency domain information and introduces an efficient diffusion mechanism to accelerate the restoration process. Experimental results demonstrate that our dual-branch model significantly outperforms state-of-the-art methods, including FBCNN, DDRM, and DOGNet, in terms of PSNR, MS-SSIM, LPIPS, FSIM, and FID metrics, achieving superior image restoration quality. These findings highlight the importance of combining pixel and frequency domain information for effective artifact removal and detail preservation.