Scholarlink

<p>As deep artificial neural networks (ANNs) get bigger, deeper, and used in more challenging applications, the need for non-gradient based training methods becomes more desirable. This paper explores a new non-gradient-based method to train ANNs and deep ANNs, the Continuous Action Learning Automata (CALA) optimizer. The CALA optimizer assigns a Learning Automata agent to every weight in a neural network and uses game theory to coordinate actions of the agents. We show that the CALA optimizer is computationally efficient, that it converges to a desired error rate faster than current gradient-based methods like stochastic gradient descent (SGD) and show how one could use a Finite Action Learning Automata (FALA) algorithm to find optimal values for the hyper-parameters required to optimize the CALA controller. The CALA method contrasts itself against other non-gradient methods in that it approaches the computational efficiency of top gradient descent methods like SGD. The CALA method converges fast, and there is any easy-to-follow algorithm to tune the hyper-parameters of the algorithm. These advantages address weaknesses that other non-gradient methods suffer from. Therefore, the CALA controller has the potential to see far greater implementation than other non-gradient-based optimization methods for training deep ANNs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimizing a continuous action learning automata (CALA) optimizer for training artificial neural networks

  • 期刊论文

  • 发表日期：

    2025年2月6日

查看全文
  • James Lindsay,
  • Sidney Givigi

摘要

As deep artificial neural networks (ANNs) get bigger, deeper, and used in more challenging applications, the need for non-gradient based training methods becomes more desirable. This paper explores a new non-gradient-based method to train ANNs and deep ANNs, the Continuous Action Learning Automata (CALA) optimizer. The CALA optimizer assigns a Learning Automata agent to every weight in a neural network and uses game theory to coordinate actions of the agents. We show that the CALA optimizer is computationally efficient, that it converges to a desired error rate faster than current gradient-based methods like stochastic gradient descent (SGD) and show how one could use a Finite Action Learning Automata (FALA) algorithm to find optimal values for the hyper-parameters required to optimize the CALA controller. The CALA method contrasts itself against other non-gradient methods in that it approaches the computational efficiency of top gradient descent methods like SGD. The CALA method converges fast, and there is any easy-to-follow algorithm to tune the hyper-parameters of the algorithm. These advantages address weaknesses that other non-gradient methods suffer from. Therefore, the CALA controller has the potential to see far greater implementation than other non-gradient-based optimization methods for training deep ANNs.