Taking an ultra-low energy consumption building in Zhengzhou City as the case study, a data-driven prediction model based on the physics-informed neural network (PINN) algorithm combined with physical information was established to realize the short-term prediction of energy consumption for the ground source heat pump (GSHP) system. Firstly, four operating strategies, which include independent fresh air system, fresh air with ground source bypass, fresh air with ground source heat pump, and fresh air coupled fan coil system, were simulated via EnergyPlus, so as to obtain the database for further modeling. The reliability of the simulation model was verified, and the average errors between the simulated indoor temperature and energy consumption values of each system and the measured values were within a reasonable range. Secondly, a resistance–capacitance (RC) thermodynamic model based on the building physics information was constructed by identifying the unknown parameters. Then, the model RC thermodynamic model was incorporated into the loss functions of neural networks to establish the data-driven PINN model, which enables the positive functions of physical information, rather than the single numerical fitting approximation in pure data-driven models. Results of comparisons show that the prediction model combined with RC model is superior to the traditional pure data-driven methods in terms of both the accuracy and the generalization ability. This study with short-term prediction can provide the technical support for further energy analysis, energy-saving optimization, and intelligent operation and maintenance of ultra-low energy buildings.