<p>Deep learning-based segmentation models have gained significant focus in various computer vision applications, including remote sensing and medical imaging. There exist deep learning architectures for semantic and instance segmentation separately, with limitations prevailing like imprecise boundary delineation, poor spatial consistency, improper fine-grained object separation, and inaccurate instance segmentation, particularly while handling intricate object structures in remote sensing images. To mitigate the aforementioned issues, in the present work, we propose a unified deep framework that integrates both semantic and instance segmentation within a single architecture tailored for high-resolution RSIs. Our framework combines an Improved Attention Residual MobileNetV2 U-Net (IARUMV2) for pixel-level semantic segmentation and Dynamic Mask R-CNN for instance-level segmentation. To further refine spatial coherence and boundary delineation, we incorporate the post-processing technique like Conditional Random Fields (CRF) on the output segmentation map of enhanced U-Net to improve spatial consistency and edge sharpness. This refined semantic mask serves as input to the Dynamic Mask R-CNN model for instance segmentation, where the Graph-based Refinement Module (GRM) is employed to improve boundary accuracy by leveraging graph-based smoothing techniques. Our approach ensures improved object delineation, increases the segmentation accuracy, and decreases false positives compared to conventional deep learning architectures. Evaluation outcomes on standard datasets illustrate that the proposed approach attains superior performance, highlighting its effectiveness in both semantic and instance segmentation tasks. The results validate the effectiveness of jointly modeling semantic and instance-level information, providing a more comprehensive understanding of complex remote sensing scenes.</p>