Dynamical system state estimation and parameter calibration problems are ubiquitous across science and engineering. Bayesian approaches to the problem are the gold standard as they allow for the quantification of uncertainties and enable the seamless fusion of different experimental modalities. Current techniques such as Kalman, particle, or variational filters apply to discretized dynamical systems. To apply these methods, practitioners must introduce fictitious transition probabilities that might lead to unsatisfactory inference bias. To address these drawbacks, this chapter aims to develop a unifying paradigm that seamlessly combines measurement data and physics to solve dynamical system state and parameter estimation problems. Our method builds upon the information field theory, which is essentially Bayesian statistics for physical fields. Specifically, we construct a physics-informed prior probability measure on the function space of system responses so that functions that satisfy the physics are more likely. This prior allows us to quantify model-form errors. We connect the system’s response to observations through a probabilistic model of the measurement process. Bayes’ rule gives the joint posterior over the system responses and all parameters. We apply and compare two numerical methods to sample from the analytically intractable posterior: (1) sampling using stochastic gradient Langevin dynamics and (2) stochastic variational inference developed in our previous works. Our approach can quantify model-form uncertainties without requiring any numerical solver. The developed methodology offers a powerful framework for Bayesian estimation in dynamical systems.