In recent years, data-driven machine learning techniques have made significant contributions to asset pricing, which uses factor models to estimate croSS-sectional expected returns. However, learning effective models from non-stationary and noisy financial data still remains a challenge. In this paper, we use variational mode decomposition (VMD) to construct observable characteristics that measure the unobservable dynamic loadings from stock price-volume data. Furthermore, we adopt a conditional variational autoencoder (VAE) architecture to extract low-dimensional factors and time-varying loadings by introducing the characteristics. Compared with the standard FactorVAE, our two-stage framework can improve the RankICIR metric by 27.8%,which represents higher predictive accuracy. Empirical tests on Chinese stock market also confirm the efficiency of our method.