Scholarlink

Echo State Networks, a lightweight approach within the reservoir computing paradigm, offer an efficient solution for processing temporal data under resource constraints. Despite their efficiency, standard architectures depend on dense output layers, which limit interpretability, increase inference cost, and hinder deployment in communication-restricted scenarios. This work, for the first time, explores a smooth sparsification method for the readout layer based on Smooth-L1 regularization, which enables efficient, unconstrained gradient-based optimization while promoting sparsity. Evaluated on datasets encompassing regression and classification tasks, SmoothL1-ESN achieves essential readout weight sparsity while maintain competitive performance compared to baseline model. These results confirm the method’s ability to balance sparsity and performance on diverse datasets, making it well-suited for scenarios demanding efficiency and communication reduction, such as distributed or federated learning systems. The code is publicly available at https://github.com/oroboro235/SmoothL1_ESN .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

On the Use of Smooth-L1 Approximation in Echo State Networks for Sparse and Efficient Temporal Modeling

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Gengcheng Lyu,
  • Manuel Röder,
  • Frank-Michael Schleif

摘要

Echo State Networks, a lightweight approach within the reservoir computing paradigm, offer an efficient solution for processing temporal data under resource constraints. Despite their efficiency, standard architectures depend on dense output layers, which limit interpretability, increase inference cost, and hinder deployment in communication-restricted scenarios. This work, for the first time, explores a smooth sparsification method for the readout layer based on Smooth-L1 regularization, which enables efficient, unconstrained gradient-based optimization while promoting sparsity. Evaluated on datasets encompassing regression and classification tasks, SmoothL1-ESN achieves essential readout weight sparsity while maintain competitive performance compared to baseline model. These results confirm the method’s ability to balance sparsity and performance on diverse datasets, making it well-suited for scenarios demanding efficiency and communication reduction, such as distributed or federated learning systems. The code is publicly available at https://github.com/oroboro235/SmoothL1_ESN .