ACT: Action-assoCiated and Target-Related Representations for Object Navigation

Object navigation tasks require an agent to find a target in an unknown environment based on its observations. Researchers employ various techniques, such as extracting high-level semantic information and building a memory network, to enhance the perception and understanding of the environment. However, these methods neglect the correlation between the representations of the current scene and the target description, as well as the relationships between perception and actions. In this paper, we propose a model that uses semantic features of the visual observation as input for navigation, which are represented in the modality similar to that of the target embedding. On this basis, we fuse the visual features and spatial masks with an Encoder-Decoder transformer structure to reflect the association between perception and actions. Furthermore, in the memory module, this paper integrates the representations of explored scenes and the target information for more direct guiding of impending navigation direction selection. Our method enables the agent to perceive the position of the target more quickly and execute accurate actions to approach it. Our method outperforms the state-of-the-art (SOTA) models in the AI2Thor environment with higher navigation success rate and better learning efficiency.