Evaluating the Performance of Different Optimizers for Deep Learned Finite Elements
摘要
The training process of deep neural network (DNN) involves adjusting numerous parameters to minimize the prediction error. Several optimization algorithms, such as stochastic gradient descent (SGD), adaptive gradient algorithm (Adagrad), root mean squared propagation (RMSProp), adaptive moment estimation (Adam), and adaptive moment estimation with infinity norm (Adamax), among others, play a pivotal role in refining network weights and biases. Each algorithm exhibits unique characteristics, affecting convergence speed and final performance. The choice of optimizer often depends on the nature of the problem, dataset size, and network architecture. The present study assesses the performance of these optimizers based on convergence analysis.