Accurately distinguishing between invasive carcinoma (IC) and carcinoma in situ (CIS) in breast histopathology is essential for diagnosis and staging, and can inform treatment planning. As digital pathology workflows increasingly integrate AI-based tools, automating this distinction in whole slide images (WSIs) remains a critical challenge due to the subtle morphological differences and the limited availability of detailed annotations. In this work, we present a multi-class segmentation framework that addresses the segmentation of IC, CIS and benign tissue in hematoxylin and eosin-stained WSIs of core needle biopsies. The model integrates a U-Net decoder with a CTransPath encoder–a transformer-based network pretrained on millions of histopathology image patches–designed to capture both local cellular features and global tissue context. A spatial refinement step is included to reduce misclassification between IC and CIS. The model is fine-tuned on a dataset of 92 WSIs with detailed expert annotations and evaluated at both tile and whole-slide levels. It outperforms baseline models and a non-finetuned version, achieving a macro-averaged tile-level Dice score of 0.80 on the test set. IC is segmented with high accuracy (Dice: 0.77), while CIS remains more difficult (Dice: 0.68), reflecting its lower prevalence and more subtle appearance. These results highlight the potential of domain-specific pretrained transformers for robust lesion-level segmentation in breast cancer under real-world constraints, with applications in diagnostic support and case prioritization.