This paper studies the problem of robot local navigation based on Twin Delayed Deep Deterministic policy gradient (TD3). TD3 achieves efficient path planning and obstacle avoidance in navigation tasks, but suffers from the limitations of slow convergence, insufficient generalization ability and susceptibility to the local optimum. Therefore, this paper proposes a novel Guidance TD3 (G-TD3) framework that integrates Temporal Convolutional Random Network Distillation (TC-RND) technology with a novel Guidance Network (GN). TC-RND captures temporal state features to infer the intrinsic reward of the current state, effectively enhancing the agent’s exploration of areas with high novelty. The GN is further developed to target the limitation of TC-RND. It generates guidance actions that direct the agent’s attention toward areas with high cognitive uncertainty and Q-value, accelerating training convergence and alleviating the local optimum phenomenon. The dual mechanism effectively improves the algorithm’s generalization capacity and robustness. Comprehensive experiments demonstrate that G-TD3 achieves substantial performance improvements across multiple environmental indicators, outperforming the state-of-the-art baseline (TD3) and its enhanced variants. The convergence time decreases by 61% and the local optimum phenomenon is reduced by about 80% compared with TD3.