<p>Traversing through unknown narrow gaps poses a significant challenge in the field of unmanned aerial vehicle (UAV) planning. The majority of previous work in this domain has been predominantly state-based, often requiring prior knowledge about the narrow gaps. Recently, the incorporation of reinforcement learning (RL) methods has shown promise. However, dealing with state-based SE(3) (special Euclidean group in 3D space, describing rigid-body motions including translation and rotation) planning problems using RL faces two main challenges: the constrained solution space inherent in SE(3) problems presents sampling difficulties during training; the sim-to-real gap in dynamic domains complicates the transfer of learned policies to real-world scenarios. To address these challenges, we propose a novel end-to-end system for quadrotor SE(3) planning. We develop a customized differentiable simulator coupled with a point-mass-thrust-vector model, enabling policy optimization through first-order gradients. We validated our algorithm in a simulation environment and conducted extensive ablation studies to further demonstrate the effectiveness of our approach.</p>