4D occupancy forecasting is a task to forecast the occupancy map at the future time from the history data. It serves as a core component of 3D situational awareness in autonomous driving, beneficial to trajectory planning in the high-dynamic environment. However, how to effectively model the 4D tendency of occupancy maps is still an open problem. In this paper, we find ordinary differential equation (ODE) is a powerful tool in modeling temporal relations, and propose a neural ODE-based 4D occupancy forecasting method as Op-Occ-Net. Firstly, we develop a theoretical framework to model the 4D occupancy forecasting from the perspective of neural ODE. Its core is to estimate the ODE operator. After that, we design a neural network Op-Occ-Net to implement this framework. In Op-Occ-Net, we derive the least-square solution of the ODE operator in the idea case and then design a Mamba-based fusion module to learn the ODE operator. Besides, as forecasting is a kind of artificial intelligence generated content (AIGC) task, we apply vector quantized variational autoencoder (VQ-VAE) to improve the learning of the ODE operator. Extensive experiments on the public Occ3D dataset indicate that Op-Occ-Net has superior intersection-over-union (IoU) and mean IoU (mIoU) metrics than state-of-the-art methods. Thus, we believe Op-Occ-Net benefits the safety of autonomous driving.