In this paper, a switching filter based on adaptive rank for salt and pepper noise removal from color medical images is presented. Color medical images are widely used to detect and diagnose diseases. But image quality is frequently degraded by noise, and such images are not suitable for the purpose. So, eliminating noise is a crucial stage in the processing of images. In this paper, for detecting impulse noise, an adaptive rank is utilized to classify a pixel as a noise candidate or not. The center pixel is probably noisy in the first step if its rank is more than the calculated rank according to the contamination level. This probably noisy pixel is further checked in the second stage to determine if it is an edge or impulse. The vector median filter’s output is used to substitute pixels that are found to be noisy. The simulation results demonstrate that the proposed filter outdoes other state-of-the-art filters in removing and retaining image details.