This paper proposes a novel macroscopic approach for forecasting the motion of a crowd in a few seconds time horizon, conditionally to a handful of past visual observations. We leverage a compact, grid-based representation of the crowd through macroscopic properties (local density and velocity) and a conditional diffusion model to generate samples of the predictive distribution for the crowd motion. We propose new evaluation metrics for this problem and compare the performance of several variants of the proposed approach on the ATC dataset.