<p>Reservoir computers, using recurrent neural networks with fixed random connections, are known to perform a wide range of information-processing tasks. Yet the transformations taking place within the reservoir, the interaction between input matrix, reservoir, and readout layer, and the influence of key design parameters remain insufficiently understood. Here, we shift the focus from performance maximization to the identification of minimal computational requirements for different model tasks. We investigate how many neurons and how much nonlinearity are needed to solve specific tasks, including cases with non-sigmoidal activation functions. Our results show that the division of labor between input matrix, reservoir, and readout layer depends strongly on the task. In many cases, a weakly coupled and only minimally nonlinear reservoir proves sufficient. In addition, features often considered secondary, such as the structure of the input matrix or the steepness of the activation functions, can become decisive for performance.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Illuminating the black box of reservoir computing

  • Claus Metzner,
  • Thomas Kinfe,
  • Andreas Maier,
  • Achim Schilling,
  • Patrick Krauss

摘要

Reservoir computers, using recurrent neural networks with fixed random connections, are known to perform a wide range of information-processing tasks. Yet the transformations taking place within the reservoir, the interaction between input matrix, reservoir, and readout layer, and the influence of key design parameters remain insufficiently understood. Here, we shift the focus from performance maximization to the identification of minimal computational requirements for different model tasks. We investigate how many neurons and how much nonlinearity are needed to solve specific tasks, including cases with non-sigmoidal activation functions. Our results show that the division of labor between input matrix, reservoir, and readout layer depends strongly on the task. In many cases, a weakly coupled and only minimally nonlinear reservoir proves sufficient. In addition, features often considered secondary, such as the structure of the input matrix or the steepness of the activation functions, can become decisive for performance.