<p>Recently, 3D object detection from LiDAR point clouds has received increasing attention in autonomous driving. Many existing methods have achieved significant performance improvements by adopting the propose-refine two-stage framework, but most of them do not fully utilize 3D structural context information. In view of this, this paper proposes a simple but effective 3D detection framework with two novel designs. Firstly, we introduce the homotopy manifold sparse convolution backbone (HMSCB) to enhance the extraction of 3D structural context features by incorporating a geometry enhancement branch. Secondly, we propose the coarse-to-fine refinement module (CFRM), which introduces the 3D structural context features in a coarse-to-fine order and gradually corrects the initial proposals generated by the region proposal network (RPN). Specifically, in the refinement process, we introduce a spatial attention mechanism to facilitate information exchange between refinement steps and design the refinement feature extraction module (RFEM) to elevate the resulting feature representation ability by decoupling the 3D structure context information. On the KITTI car 3D detection validation and test sets, our method achieves an average precision (AP) of 88.11% and 82.05% at the moderate difficulty level, respectively, and on the challenging multi-category Port dataset, it achieves 69.1% mean average precision (mAP), showcasing the effectiveness of the proposed method.</p>