Computer-aided diagnosis (CAD) has become an essential solution for breast ultrasound (BUS) image analysis; however, the development of CAD systems is hindered by high-quality data scarcity and annotation challenges. We propose a novel clinical prior-guided tumor generation method that allows precise control over tumor characteristics, such as size, shape, and texture, using clinical knowledge from textual descriptions and structural masks. Additionally, our method enables cross-domain data generation, enhancing the adaptability of the synthetic data across different imaging conditions. Experiments on three public BUS datasets demonstrate the favorable generation quality and effective cross-domain adaptation of our method. Moreover, the improved accuracy in downstream classification and segmentation tasks further show the clinical utility and practical effectiveness of our synthetic images in supporting breast cancer diagnosis. The code is available at https://github.com/Violetphy/Clinical-Prior-Tumor-Generation .