Scholarlink

This work presents a contrastive self-supervised learning framework for biomedical text classification, leveraging article–abstract alignment from the PubMed Article Summarization Dataset. By treating each article and its abstract as a positive pair, and other abstracts in the batch as negatives, transformer models learn semantically aligned representations without requiring manual labels. The approach is evaluated on multi-class classification tasks across biomedical categories using BERT-base, BioBERT, and PubMedBERT. Results show that the proposed method achieves an average F1-score of 87.4% and AUC of 0.91, outperforming BERT-base (F1 82.1%, AUC 0.86) and randomly initialized baselines (F1 68.5%, AUC 0.74). Gains are especially pronounced in low-resource and class-imbalanced settings. These findings demonstrate that domain-aware contrastive pretraining offers a scalable solution to biomedical document classification, enhancing both robustness and predictive confidence while reducing dependence on annotated data.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Contrastive Self-Supervised Pretraining for Biomedical Text Classification

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Orestis Papadimitriou,
  • Ioannis Karamitsos,
  • Khalil Al-Hussaeni,
  • Vassilis C. Gerogiannis,
  • Manolis Maragoudakis

摘要

This work presents a contrastive self-supervised learning framework for biomedical text classification, leveraging article–abstract alignment from the PubMed Article Summarization Dataset. By treating each article and its abstract as a positive pair, and other abstracts in the batch as negatives, transformer models learn semantically aligned representations without requiring manual labels. The approach is evaluated on multi-class classification tasks across biomedical categories using BERT-base, BioBERT, and PubMedBERT. Results show that the proposed method achieves an average F1-score of 87.4% and AUC of 0.91, outperforming BERT-base (F1 82.1%, AUC 0.86) and randomly initialized baselines (F1 68.5%, AUC 0.74). Gains are especially pronounced in low-resource and class-imbalanced settings. These findings demonstrate that domain-aware contrastive pretraining offers a scalable solution to biomedical document classification, enhancing both robustness and predictive confidence while reducing dependence on annotated data.