As online communication becomes increasingly complex, fine-grained inappropriate speech detection has become increasingly important in content moderation. To address challenges such as ambiguous category boundaries and imbalanced data distribution, this paper proposes a detection model called MoCoBERT, based on momentum contrastive learning. The model integrates a BERT pre-trained language model with a momentum-based contrastive learning mechanism, enhancing the discriminative power of text representations through the construction of semantic positive and negative sample pairs. Additionally, a hybrid loss strategy combining Focal Loss and feature-level Mixup is employed to improve detection performance on minority classes. A multi-task learning framework is also adopted to enhance the model’s robustness. We also construct a large-scale Chinese fine-grained inappropriate speech dataset (CFGIS) with 92,048 labeled samples. Experimental results demonstrate that MoCoBERT significantly outperforms existing baseline models across various evaluation metrics.