Change Detection (CD) on remote sensing images plays a crucial role in monitoring surface dynamics. However, current CD methods still suffer from several challenges, including pseudo changes and large variation in object scales. In this paper, we propose a novel Difference-Guided Hybrid Attention Network (DHANet) to address the above challenges. Specifically, we introduce the Neighbor Fusion Module (NFM) to aggregate semantic features from adjacent scales within the Siamese backbone, thereby enhancing the features representation capability. Additionally, by integrating both self-attention and cross-attention mechanisms into a unified architecture and with the difference features serving as the guidance, we propose the Difference-Guided Hybrid Attention Module (DHAM), which regulates the global distribution of features, promotes intralevel representation interaction and filters out irrelevant information. We further design the Multiscale Perception Module (MPM), which employs multiscale asymmetric convolutions and dilated convolutions to extract object features of different scales and capture complex contextual information. Extensive experiments demonstrate our method outperforms ten SOTA CD methods on three widely used CD datasets.