Embodied navigation, which involves robotic agents exploring an unknown environment to reach target locations with egocentric observation, is a complex problem in the field of embodied AI. Audio-visual navigation extends this concept by equipping agents with both visual and auditory sensors. Recent studies have explored the audio-visual navigation task, exploring its potential and intricacies. Current methodologies, despite their achievements, often fail to fully utilize the capabilities of these sensory modalities, leading to sub-optimal designs and inefficiencies. In addition, the prevalent backtracking problem in navigation tasks often leads to redundant movements by robots. To address these challenges, we introduce the Audio-Visual Guided Navigation (AVGN) model. At its core, AVGN champions the fusion of visual and auditory data through sophisticated modality fusion layers. Our model represents a significant advancement by employing transformers for visual encoding and deploying a novel Dual Stage Feature Integration (DSFI) to decode latent interrelationships between the visual and audio realms. With AVGN, the backtracking issue is mitigated, as the acoustic map synergizes more effectively with visual features for informed action decisions. Furthermore, targeting the backtracking problem, we propose a unique set of reward structures to guide and refine the actions of the agent. Benchmark evaluations on Replica and Matterport3D datasets validate our claims, and AVGN notably surpasses existing methodologies in audio-visual navigation tasks.