Breast magnetic resonance imaging (MRI) is widely recognized for its high sensitivity in detecting breast cancer. However, interpreting breast MRI scans remains a complex, time-consuming, and resource-intensive task, even for experienced radiologists. To address these challenges, artificial intelligence-based methods are increasingly being employed. In this study, we developed a multimodal breast MRI language-image pre-training (MLIP) approach as an initial exploration of a breast MRI foundation model to aid in the interpretation of scans. Two types of inferences were used to evaluate MLIP’s performance. First, MLIP could retrieve corresponding MRI cases from a dataset based on a query, achieving an area under the receiver operating characteristic curve of 0.717 for suspicious and malignant cases, 0.640 for dense breasts, and 0.601 for low background parenchymal enhancement (BPE). Second, MLIP demonstrated the ability to predict the level of disease suspicion for a given MRI case. The results suggest that MLIP has the potential to serve as a foundation model for breast MRI interpretation. Future work will focus on expanding its capabilities through various downstream tasks and integrating additional models to enhance overall performance.