Semantic segmentation of Multiple Sclerosis (MS) lesions in longitudinal MRI is essential for tracking disease progression. This study evaluates how well various deep learning segmentation models, commonly used in medical imaging, generalize when integrated into a diffusion model framework. We conduct extensive experiments testing different architectural configurations and inference strategies to identify optimal setups for MS lesion segmentation. In particular, we examine how combining outputs from multiple diffusion time steps affects performance and robustness. Results show that certain backbone architectures significantly enhance performance, and that appropriate inference strategies can further improve segmentation accuracy. These findings highlight the potential of diffusion-based methods for clinical MS analysis and offer guidance on model and inference selection to achieve reliable lesion segmentation from MRI scans. Our work contributes to the growing body of research applying generative models to medical imaging, especially in the context of progressive diseases like MS.