Deep learning for inverse problems in medical imaging is frequently limited by scarce annotated data and inherent underdetermination. We propose a novel, physics‐informed framework that leverages large‐scale video data to generate 3D training volumes. Our method converts video sequences into surrogate training data by aligning intensity distributions, simulating realistic noise, and extracting dynamic patches that capture diverse structural features. Concurrently, we introduce a composite loss function that integrates intensity similarity, physics consistency, edge‐aware, and structural similarity losses to enforce data fidelity and physical plausibility. We validate our approach on the ill‐posed field‐to‐susceptibility inversion problem in quantitative susceptibility mapping. This integrated strategy offers a promising solution to overcome data scarcity and enhance deep learning for challenging inverse problems in medical imaging.