Segmenting Key Clues to Induce Human-Object Interaction Detection

Two-stage HOI detectors have made great progress in training and inference, but still suffer from loss of original image features and ambiguous human-object relationships. To address the above issues, this paper proposes a network based on segmentation to extract key features to induce human-object interaction detection, which consists of two parts. First, a segmentation-based module is designed to extract fine-grained interaction features from original images, which is then refined by a feature learning encoder. Secondly, a graph-based module is proposed to encode the spatial relationships of detected human and object instances, which is to learn the interaction contexts from pair-wised human-object contexts. A transformer decoder is then utilized to equip the interaction features from the original images with the interaction contexts. The proposed method can directly learn fine-grained interaction features under the guidance of spatial relationships, achieving state-of-the-art performance on two standard benchmarks for HOI detection, HICO-DET and V-COCO.