Decision-based black box attacks are rising concerns in adversarial machine learning, as they allow attackers to manipulate the outputs of machine learning models without having access to the model’s internal architecture or hyperparameters. Sparse attacks, aiming to minimize the number of perturbed pixels, expose critical vulnerabilities in machine learning models, representing a considerable threat to real-world systems. A current limitation of sparse attacks is the need to query the target model in the range of thousands of queries to create imperceptible adversarial examples, which can be costly and easily detected. Our study demonstrates the potential of using the patch-wise adversarial removal (PAR) algorithm to improve the query efficiency of sparse attacks. To defend against sparse decision-based attackers, we find that adversarial training is an effective countermeasure, strengthened further by using median filtering and adversarial detection. We probe the possibility of enhancing the attacks with our modification of the PAR algorithm, blurring the adversarial example with the original unperturbed input, with results showing that the F1-score of the trained detector drops from 0.97 to 0.89. The study highlights the importance of continued research into understanding the potential severity of sparse attacks and optimizing related defenses.