In response to the escalating demand for Content Delivery Network (CDN) website classification, this paper introduces Multi-Modal Multi-Task Tiered Expert(M3TTE), a revolutionary method designed to enhance the efficiency and accuracy of different classification tasks of CDN websites. Facing the limitations of existing models, M3TTE leverages a tiered expert network, employing specialized experts for processing different modal features. This tiered structure ensures personalized treatment, addressing the challenges posed by diverse modalities. The framework is anchored in four key principles: personalized handling of features from different modalities, retention of each modality’s specificity, integration of expert outputs for multi-task learning, and preservation of both global and task-specific features. M3TTE’s architecture achieves a delicate balance between global and task-specific information, showing remarkable performance in CDN website classification. Experiments, conducted on a diverse dataset collected from various devices and operating systems, underscore M3TTE’s superiority over single-task and multi-task models. Impressively, M3TTE attains 74.93% accuracy and 74.89% F1 score for website category classification and 97.48% accuracy and F1 score for CDN classification. This paper presents M3TTE as an excellent solution, effectively tackling the complexity of CDN website classification through rigorous experiments and evaluations.