More than one-third of the world population suffers from allergy symptoms, thus pollen monitoring is performed worldwide, to provide data on pollen seasons for people with allergies. To this end, Hirst traps that catch the airborne pollen grains and other particles are often used, and then specialists count the pollen grains of each taxon under microscope. This is a tedious task, so we would like to automate recognition and counting using deep learning-based object detectors. In this work, we investigate how changing the color representation affects pollen grain detection in images. Five different representations were examined: RGB, AvgRGB, STRESS, CMYK, and Magenta, which we believe may be particularly useful, as pollen is dyed pink with fuchsin to improve visibility under a microscope. The average precision results for the investigated detectors are above 98% when both training and test data come from the same camera. However, precision decreases when the test data come from a different camera. We observed that only the conversion of images to Magenta allows for high pollen detection precision in images from a different camera than the one used to capture the training samples.