Visual-language navigation (VLN) requires the agent to understand language instructions and navigate in unseen environments. The agent needs to align language instructions with real-time and historical features to achieve goal grounding. Existing methods struggle to align instructions with the environment and rely on rigid navigation processes, limiting performance in unseen environments. This stems from the disconnect between instructions and navigation processes, as well as the weak correlation between the endpoint and instructions, which interferes with navigation. Thus, we propose a Process-Adaptive cRoss-modal Transformer (PART), which dynamically adjusts the navigation history based on instructions and links real-time action prediction with memory reasoning. This approach enhances the alignment between instructions and environments while mitigating the adverse effects of overlapping trajectories. Additionally, PART uses a distance-adaptive loss function to reduce reliance on specific trajectories while reinforcing goal-directed learning, enhancing generalization in unseen environments. On the goal-oriented VLN benchmark REVERIE and the step-by-step VLN benchmark R2R, PART surpasses previous state-of-the-art methods.