Due to emotional ambiguity and the subjectivity of annotators, noisy labels have become an unavoidable problem in facial expression recognition (FER). Most existing methods primarily employ supervised learning to address the noisy labels, which still tends to overfit them. In this paper, we propose a novel noise-tolerant FER model that integrates the prototypical classifier and the general classifier within a consistency-driven framework, to address the overfitting to noisy labels. In the proposed method, a prototypical classifier is designed based on emotional category-feature prototypes, replacing fully connected layers with a distance metric-based classification head. Furthermore, to eliminate emotional ambiguity in facial images, we propose an emotion consistency constraint between different classifiers, complementing our model to achieve the correction of noise labels, and meanwhile leveraging classifiers with varied metrics for improved detection and correction of noisy labels. Finally, experimental results illustrate that the proposed method outperforms the state-of-the-art methods on FER datasets with various noises.