Accurate and abundant annotation of mammographic images is of the utmost importance to train robust and reliable artificial intelligence (AI) models for improving the detection of breast cancer. However, manual annotation of these complex medical images is a significant bottleneck, characterized by its tedium, high time requirement from expert radiologists, and the inevitable presence of inter-observer discordance and errors. This paper proposes a novel, very efficient semi-automatic annotation system that leverages the strengths of both worlds, i.e., AI capabilities and human knowledge, to accelerate the annotation process without any loss in annotation quality. Our method pursues a two-stage strategy: Firstly, a pre-trained, big dataset CNN model, fine-tuned on a small set of manually annotated Moroccan women’s mammograms, generates initial, pixel-level segmentation masks for non-annotated images. These segmentations generated by AI are then validated, reviewed, and enhanced by senior radiologists through an interactive interface, and the errors are easily corrected. These enhanced annotations are then fed back to the CNN for continuous training and performance enhancement, creating an active learning loop that continues to reduce the need for manual input. Experimental findings demonstrate that this semi-automatic framework achieves a remarkable reduction in annotation time, along with improved annotation consistency and more accurate boundary delineation, leading to an enhanced quality of labeled data. Not only does this approach provide an immediate solution to the shortage of annotated mammographic images, it provides an effective and scalable solution for accelerating the development and deployment of AI-based diagnostic software, paving the way for broader access to better breast cancer screening.