Disparity of Abstract Color Representations in Convolutional Networks

A common occurrence in the deep learning community is to theorize on what abstract representation data takes inside a network. The aim of this paper is to demonstrate that the expected representation from a simple well-defined problem takes on other forms than what human thought processes would expect it to do. The experiment uses CIFAR-10 and a FRUITS dataset as its base and compares models trained on RGB variants to models trained on separated red, green, and blue color channels. A simplified FRUITS dataset variant represents a much simpler problem of classifying tomatoes and capsicum based on their significant red and green color. Generally, the RGB model variant outperformed the separated variants; but for the simplified FRUITS experiment, we observed the blue color channel outperforming the red and green model variants with its performance on par with the RGB variant. This suggests that the model has chosen a blue filter to represent the classification features. We also confirm that the blue color variant is most prominent feature contributor through cross-testing the RGB model variant. For a simple classification problem, the models choose to represent the classes in a less intuitive form than what was expected from the simplicity of the data representation. We aim for further discussion about the disconnect of internal data representation of deep learning to the human counterpart.