Learnable Context in Multiple Instance Learning for Whole Slide Image Classification and Segmentation

Multiple instance learning (MIL) has become a cornerstone in whole slide image (WSI) analysis. In this paradigm, a WSI is conceptualized as a bag of instances. Instance features are extracted by a feature extractor, and then a feature aggregator fuses these instance features into a bag representation. In this paper, we advocate that both feature extraction and aggregation can be enhanced by considering the context or correlation between instances. We learn contextual features between instances, and then fuse contextual features with instance features to enhance instance representations. For feature aggregation, we observe performance instability particularly when disease-positive instances are only a minor fraction of the WSI. We introduce a self-attention mechanism to discover correlation among instances and foster more effective bag representations. Through comprehensive testing, we have demonstrated that the proposed method outperforms existing WSI classification methods by 1 to 4% classification accuracy, based on the Camelyon16 and the TCGA-NSCLC datasets. The proposed method also outperforms the most recent weakly supervised WSI segmentation method by 0.6 in terms of the Dice coefficient, based on the Camelyon16 dataset.