In autonomous driving perception, millimeter-wave radar is highly regarded for its all-weather and interference-resistant performance. However, its application is constrained by high noise levels, sparse semantic information, and the substantial computational overhead and insufficient real-time performance of existing object detection methods. To address these challenges, we propose MF-Radar, a U-shaped, spectrum-enhanced Mamba network. First, the Multi-branch Attention Separable (MAS) Module employs multi-scale spatial feature fusion and channel-wise attention to effectively preserve and refine the spatial topology and feature representations of 3D targets. Next, within the Enhanced Mamba (EM) Block, the core FFT-Enhanced Mamba (FE-Mamba) component utilizes a dual-channel collaborative mechanism across temporal and frequency domains to achieve deep fusion of long-range dependencies and spectral features. Experimental results demonstrate that MF-Radar outperforms current state-of-the-art methods on both the CRUW and CARRADA datasets, while reducing model parameter count by approximately 75% and Giga Floating-point Operations Per Second (GFLOPs) by approximately 72%, thereby validating its efficiency and strong generalization capability. The source code will be released at https://github.com/RAASDAD/MF-Radar.git .