Glioblastoma (GBM), primary central nervous system lymphoma (PCNSL), and brain metastases (BM) are three common malignant central nervous system tumors. Accurate preoperative differentiation is essential for appropriate treatment planning and prognosis, however, it’s challenging to differentiate these tumors using MRI due to their similar anatomical structures and imaging characteristics. In this paper, we first construct a new multi-center brain MRI dataset, including 315 training cases (GBM 64, PCNSL 59, BM 192) and 124 external test cases (24:23:77). Moreover, we propose a novel framework FoTNet for accurate diagnosis of the three tumors. Our model achieves a classification accuracy of 92.5% and an average AUC of 0.9754, outperforming previous methods. Our results demonstrates the great potential of AI in assisting physicians in differentiating between GBM, PCNSL, and BM, particularly in resource-limited clinical settings.