<p>Despite achieving significant results, mapless navigation still faces enormous challenges because of environmental perception errors caused by noises, uncertainties in sensor data, and the inadequate real-time performance and decision-making capabilities in dynamic environments. This paper proposes a novel deep-reinforcement-learning-based autonomous navigation framework, BMTD3, to address goal-driven navigation tasks in complex dynamic environments, with both static and dynamic obstacles, without pre-built maps. A bidirectional gated recurrent unit is employed to acquire the memory ability for the network and alleviates partly the observability problem. A multi-head attention mechanism is developed to extract different key historical features for policy generation and value estimation in navigation, improving the response ability to complex dynamic scenes, as well as the stability and accuracy in complex environments. Furthermore, a novel reward function is designed to enhance navigation accuracy and adaptability, effectively addressing the issue of reward sparsity. Simulation results validate that, compared to conventional TD3 and other baseline navigation strategies, BMTD3 achieves superior navigation success rates and enhanced robustness across both static and dynamic scenarios.</p>