Object Centric Body Part Attention Network for Human-Object Interaction Detection

The current transformer-based human object interaction (HOI) detection methods have achieved great progress, however, these methods adopt the same structre of decoder to detect human and object, which limits the accuracy of object feature extraction, thereby limiting the accuracy of HOI detection. And due to the distribution differences of multi-granularity features between human and object, the key of HOI detection is object centric interaction with the correlative human body parts. To address this issue, we propose an Object Centric Body Part Attention Network for Human-Object Interaction. First, we introduce a dual-branch decoder for human and object detection named Object Centric Decoder (OCD), where one focuses on quering objects and another pay attention to catch human who interacts with them. Secondly, in order to exploit more fine-grained human body information centered around object, we propose a Body Part Attention (BPA) module to obtain the interactive human body part features for HOI detection. We evaluated our proposed OBPA on the HICO-DET and V-COCO datasets, which significantly outperforms existing counterpart (1.7 mAP on V-COCO, and 0.9 mAP on HICO-DET compared to GEN-VLKT). Code will be available on https://github.com/zhuang1iu/OBPA-NET .