Mammogram is the gold standard for early breast cancer screening, and its integration with deep learning-based computer-aided diagnosis (CAD) models has demonstrated significant advantages in improving the accuracy of breast cancer diagnosis. However, due to differences in mammography acquisition protocols and scanner models, significant inter-domain variations exist in images obtained from different mammography devices. As deep learning models tend to overfit to domain-specific feature representations during training, models trained on source domain often experience notable performance degradation when applied to cross-domain data, hindering their deployment in dynamic clinical settings. Therefore, this paper proposes a novel domain generalization approach for mammogram classification by suppressing domain-specific features (MC-SDS). MC-SDS first employs an adaptive channel filter to identify and drop channels that have a tendency to capture domain-specific features to suppress domain-specific features. Then, by perturbing the low-frequency components, the model is encouraged to learn from the high-frequency parts, further suppressing the domain-specific features present in the low-frequency components. Experiments conducted on a public dataset and two internal datasets demonstrate that MC-SDS outperforms other benchmark methods.