Peripheral Capillary Oxygen Saturation (SpO \(_2\) ) estimation from facial videos has emerged as a promising non-invasive method for monitoring blood oxygen saturation. However, existing approaches often overlook intrinsic physiological signal properties, limiting their generalizability in real-world scenarios. This work introduces PHASE: Physiological Dynamics-Based Attention for SpO \(_2\) Estimation, a novel framework that integrates multiple innovations for enhanced accuracy and robustness. PHASE leverages a Channel Attention Transformer (CAT) to capture inter-color space dependencies across RGB, HSV, HLS, and Lab color spaces, enabling effective extraction of physiological features. Additionally, AC/DC component separation is employed to enhance signal quality by isolating pulsatile and baseline signals. To model fine-grained temporal variations, a Liquid Time-Constant (LTC) network is incorporated, providing resilience against motion artifacts and dynamic lighting conditions. Further, a novel loss function is proposed to enforce periodicity and constrain the frequency content of the extracted signals within the physiological range of heart rate dynamics, ensuring accurate SpO \(_2\) estimation that aligns with real-world physiological properties. Experimental results on the PURE and BH-rPPG datasets demonstrate that PHASE achieves state-of-the-art performance, delivering robust and accurate SpO \(_2\) predictions across diverse conditions, including varying motion and lighting environments.