Deep learning models such as Bidirectional Encoder Representations from Transformers (BERT) achieve strong performance in text classification but struggle with uncertainty, especially in ambiguous or overlapping medical categories. We present BERT-TCPE, a lightweight framework that augments BERT with Term-Class Probability Embeddings (TCPE) derived from smoothed term–class likelihoods. Unlike ensemble-based methods that add computational cost, BERT-TCPE integrates probabilistic term semantics with contextual embeddings in a simple and efficient way. Experiments on two benchmark datasets—PubMed and Ohsumed—show consistent accuracy improvements over BERT, particularly in uncertain cases, without increasing inference time or parameter size. Embedding visualizations further indicate that BERT-TCPE enhances class separation. These results demonstrate that probabilistic-semantic augmentation provides an interpretable and scalable solution for improving robustness and decision confidence in medical Natural Language Processing.