Cancer diagnosis and treatment present formidable challenges, especially with the highly variable imaging modalities required for different cancer types. Liver, brain, and breast cancers are among the most dangerous and diagnostically demanding types. Accurate subtype identification and staging necessitate meticulous analysis of cellular structures and morphological features, with Computed Tomography (CT) scans commonly employed for liver and brain cancers and ultrasound for breast tumors. The inherent variability in these imaging techniques underscores the need for advanced automated diagnostic tools to enhance accuracy and consistency. In this paper, we introduce Richard’s curve based Attention-aided U-Net (RAU-Net), a robust and adaptive model architecture designed to automate the segmentation process for liver, brain, and breast tumors. RAU-Net leverages a U-Net based structure enhanced with a unique re-parameterized Richard’s curve-based fuzzy attention mechanism. This integration enables precise feature extraction and segmentation across diverse medical imaging modalities. Our approach significantly reduces diagnostic times and improves accuracy, providing vital support for pathologists and ultimately enhancing patient outcomes. Our results indicate that RAU-Net not only improves diagnostic accuracy, but also enhances the efficiency and consistency of cancer detection and treatment planning, making it a valuable tool in clinical pathology. Our codes are publicly available at the [ GitHub repository ].