Class-Level Multiple Distributions Representation are Necessary for Semantic Segmentation

Existing approaches focus on using class-level features to improve semantic segmentation performance. How to characterize the relationships of intra-class pixels and inter-class pixels is the key to extract the discriminative representative class-level features. In this paper, we introduce for the first time to describe intra-class variations by multiple distributions. Then, Multiple Distributions Representation Learning (MDRL) is proposed to augment the pixel representations for semantic segmentation. Meanwhile, we design a class multiple distributions consistency strategy to construct discriminative multiple distribution representations of embedded pixels. Moreover, we put forward a multiple distributions semantic aggregation module to aggregate multiple distributions of the corresponding class to enhance pixel semantic information. Our approach can be seamlessly integrated into popular segmentation frameworks FCN/PSPNet/CCNet and achieve 5.61%/1.75%/0.75% mIoU improvements on ADE20K. Extensive experiments on the ADE20K, Cityscapes and COCO-Stuff datasets have proved that our method can achieve state-of-the-art performance. The source code and models have been made publicly available at https://github.com/JianJianYin/MDRL .