Gaze refers to the directed focus of an individual’s visual perception, playing a fundamental role in human communication and cognition. Recent studies have employed neural networks to predict gaze from standard RGB face images. However, obtaining effective face images is challenging due to their sensitivity to bounding box size. The interference from head pose further complicates gaze estimation, yet in real-world scenarios, it is not feasible to obtain accurate head pose values. To overcome these challenges, we design the IPHGaze network guided by head pose information for image pyramid face gaze estimation. We craft this network to capture diverse face perspectives by incorporating various face bounding box sizes, ensuring rich gaze features. Additionally, a Feature Ensemble Module (FEM) facilitates feature sharing across image pyramid levels. We use head pose features instead of precise labels in two stages: feature communication and fusion, enhancing robustness for stable gaze predictions. Our method achieves a \(5.4 \%\) improvement on EyeDiap dataset and a \(2.5 \%\) improvement on Gaze360 dataset compared with existing methods, which demonstrates its effectiveness and versatility across diverse indoor and outdoor scenarios.