Time series forecasting models often either focus on time-domain or frequency-domain, or alternatively, create network architectures to combine both. While these models achieve performance improvements, modeling time-domain and frequency-domain either solely or separately tends to underestimate the complementary relationship and intrinsic connections between them. In this paper, we explore a novel time-frequency perspective, propose a unified time-frequency analysis framework that models temporal dynamics and spectral components simultaneously via a spectrogram-based time-frequency representation (TFR). We introduce Spectrogram Band-wise Normalization to balance the distribution of frequency bands, and then introduce a Time-Frequency Enhanced (TFE) Block with two lightweight, channel-independent models, SpecLinear and SpecMLP, that learn directly from the normalized STFT spectrogram. Our models follow mainstream architectural patterns but with significantly simplified structures. Extensive experiments on multiple real-world datasets demonstrate that our TFE models outperform traditional methods, highlighting the effectiveness of a unified time-frequency perspective for designing simple yet powerful forecasting approaches.