Estimation Algorithms Based on Deep Learning

In this chapter, we shall first review the estimation problem from the perspective of the machine learning. Then we shall revisit the classical neural networks models including feedforward neural networks (FNNs) and recurrent neural networks (RNNs) in details. Specifically, we’re going to introduce the specific mathematical form of their network architecture. Then we shall discuss their approximation ability, the universal approximation theorem, and give an elegant proof. And we also introduce the mathematics on how to train the neural networks, i.e., the backpropagation algorithm and currently popular optimization algorithms for deep learning optimization problems. Finally, we shall introduce how to use deep learning method to solve state estimation problems, i.e., filtering problems.