Scholarlink

<p>This paper presents a novel framework for non-linear equivariant neural network layers on homogeneous spaces. The seminal work of Cohen et al. on equivariant <i>G</i>-CNNs on homogeneous spaces characterized the representation theory of such layers in the <i>linear</i> setting, finding that they are given by convolutions with kernels satisfying so-called steerability constraints. Motivated by the empirical success of non-linear layers, such as self-attention or input dependent kernels, we set out to generalize these insights to the <i>non-linear</i> setting. We derive generalized steerability constraints that any such layer needs to satisfy and prove the universality of our construction. The insights gained into the symmetry-constrained functional dependence of equivariant operators on feature maps and group elements informs the design of future equivariant neural network layers. We demonstrate how several common equivariant network architectures—<i>G</i>-CNNs, implicit steerable kernel networks, conventional and relative position embedded attention based transformers, and LieTransformers—may be derived from our framework.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Equivariant non-linear maps for neural networks on homogeneous spaces

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月27日

查看全文
  • Elias Nyholm,
  • Oscar Carlsson,
  • Maurice Weiler,
  • Daniel Persson

摘要

This paper presents a novel framework for non-linear equivariant neural network layers on homogeneous spaces. The seminal work of Cohen et al. on equivariant G-CNNs on homogeneous spaces characterized the representation theory of such layers in the linear setting, finding that they are given by convolutions with kernels satisfying so-called steerability constraints. Motivated by the empirical success of non-linear layers, such as self-attention or input dependent kernels, we set out to generalize these insights to the non-linear setting. We derive generalized steerability constraints that any such layer needs to satisfy and prove the universality of our construction. The insights gained into the symmetry-constrained functional dependence of equivariant operators on feature maps and group elements informs the design of future equivariant neural network layers. We demonstrate how several common equivariant network architectures—G-CNNs, implicit steerable kernel networks, conventional and relative position embedded attention based transformers, and LieTransformers—may be derived from our framework.