LiDAR scenes constitute a fundamental source of training data for numerous autonomous driving applications. However, diverse scenes with adverse weather conditions are rarely available, limiting the robustness of downstream machine learning models and the reliability of autonomous driving systems in such conditions. Collecting diverse scenes under adverse weather is challenging due to seasonal limitations. Generative models represent the current state-of-the-art in data generation, and we therefore consider them for augmenting specific driving scenarios with adverse weather conditions. In this paper, we propose a latent diffusion process combining an autoencoder and a latent diffusion model, to simulate adverse weather conditions by applying a diffusion and denoising process to clear weather scenes. We further improve realism by applying a postprocessing step to enhance the generated adverse weather scenes. We create a 3D object detection benchmark for adverse weather conditions based on a publicly available dataset and compare our augmentation method against several baselines. Our best model, trained on both the original data and our augmentation with adverse weather conditions, achieves 8.8 mAP improvement over the baseline model trained without augmentation. Code: https://github.com/matteandre/AWC-LDM .