Facial Expression Recognition (FER) is a critical task in computer vision with applications in mobile technology, education, and security. However, FER remains challenging due to imaging variations and individual differences (e.g., age and gender), leading to fine-grained classification difficulties, scale sensitivity, and significant intra-class discrepancy. To address these challenges effectively, we introduce DDANet (Dual-Path Direction-Aware Network), designed specifically for FER. Specifically, we propose the Hierarchical Directional CrossAttention (HDCA) mechanism, which is direction-aware and adept at extracting hierarchical features along horizontal and vertical axes. This is achieved through cross-attention modeling, which facilitates interaction between these two orientations, thus enhancing their directional perceptivity. Moreover, the Multi-head Grouped Coordinate Attention (MGCA) generates direction-aware and position-sensitive attention maps by discerning various regions within the network using multiple attention heads, while employing a loss function to prevent overlap in attention. Lastly, the Flip Consistency Module (FCM) is leveraged to optimize both attention and probability distributions, thereby constraining the network’s learning process to reduce intra-class discrepancy. Extensive experiments conducted on several FER datasets (RAF-DB, FERPlus, and AffectNet) demonstrate the superior performance of DDANet, confirming its effectiveness and providing robust empirical evidence for its advantages.