The agents’ ability to detect 3D objects is constrained by the limited perspective and perception range of their sensors, especially in challenging scenarios such as occlusion. Recently, multi-agent collaborative perception has demonstrated its capability to enhance detection accuracy through information exchange among different agents. However, existing multi-agent collaborative perception methods often rely on precise localization information of agents to align their sensor data, making them sensitive to localization error. To address this problem, we propose a localization-free collaborative multi-agent 3d object detection method LFCo3D. Different from existing methods, LFCo3D firstly estimates the relative-pose between agents through a novel designed Spatial Structure Matching (SSM) algorithm and a designed two-stage optimization algorithm (PSO-GO), and then aggregates detection results of different agents. Experimental results on public OPV2V and DAIR-V2X datasets demonstrate the superiority of the proposed LFCo3D.