Early recognition and treatment of breast tumors can lower death rates. As of right now, mammography is the most popular and reliable imaging method for diagnosing breast cancer. However, as it can be difficult and error-prone to identify malignant lumps from surrounding tissue, a misinterpretation of a mammogram may lead to a false diagnosis rate. We propose constructing a model based on VGG16, as it exhibits the highest accuracy. To ensure optimal accuracy and assess the resilience of VGG16, an ablation study is conducted. To reduce artifacts and improve image quality, a variety of image processing methods are used with the right parameter values. The approach has taken the INbreast dataset, which contains 7,632 images that belong to 2 classes. The model VGG16 was built on the INbreast dataset for breast cancer diagnosis. While working with a small number of complex medical images, our suggested approach, which combines image processing, transfer learning, fine-tuning, and ablation study, has shown a high accuracy rate in the classification of breast cancer. Through this research, we aim to contribute to the advancement of early breast cancer diagnosis by harnessing the adaptive and efficient characteristics of VGG16. The method has given a 96.47% accuracy for the mammogram INbreast dataset.