Breast cancer has emerged as one of the leading causes of cancer-related mortality among women globally, with early detection being critical for improving patient outcomes. While deep learning approaches show promise for automated breast ultrasound (BUS) image analysis, existing methods face significant challenges. Current CNN-based detectors struggle with the multi-scale nature of breast lesions due to static convolutions, while Transformer-based frameworks like DETR suffer from unstable query selection and poor sensitivity to object resolution variations, resulting in suboptimal detection performance for tumors with blurred boundaries. To address these challenges, we propose a Saliency-Guided selection driven Multi-Scale DEtection TRansformer network (SGMS-DETR). By leveraging Multi-Scale Dynamic convolution, our model adaptively captures breast tumor features with varying sizes and spatial distributions. A scale-invariant supervised mechanism is further introduced to extract diverse information from multi-scale features, enhancing robustness to varying image resolutions. Additionally, a Saliency-Guided Strategy that integrates global context and local details is employed to optimize the query generation process. Comprehensive experiments on both public and large-scale private datasets demonstrate that our approach outperforms existing Transformer-based methods while maintaining competitive performance with state-of-the-art detectors. Experimental results on the BUSI dataset show that our model significantly boosts detection performance, achieving a mAP50 of 82.4% and a Precision of 88.2%.