<p>Image classification with deep neural networks has reached state-of-the-art with high accuracy. The unreasonable effectiveness of deep neural networks is credited to the manifold hypothesis that states natural data lies on a low-dimensional manifold embedded in the high-dimensional space. The machine learning models learn patterns on these low-rank representations, which gives the learning algorithms robustness. We test the robustness of learning algorithms using the paradigm of “perturb and learn”. This paper proposes a novel technique called Low-Rank Perturbation Adjustment (LoPA), an implicit regularization method used by machine learning models for resisting external perturbations. LoPA exploits the dependencies in model weights that lie in high-dimensional space and projects to low-dimensional while resisting perturbation. We validate LoPA through singular value decomposition (SVD) theory and empirical experiments, showing the statistical distribution of trained model weights of zero mean and small variance. We inject perturbations into our model by hot-swapping the activation functions and interchanging loss functions during the training. An InceptionV3 Neural Network is trained on common FruitFly Drosophila images for binary classification tasks of cancer cells. The Drosophila cancer images are prepared in our lab through immunostaining protocol.</p>