Scholarlink

<p>As the core component of the transformer model, the attention has been proved as all you need in artificial intelligence field in recent years. However, conventional electronic processors are unable to cope with the exponentially increasing hardware costs and energy consumption of the computing-expensive attention. While the photonic neural network (NN) chips provide alternative energy-efficient solutions for accelerating the matrix multiplication (MM), existing photonic accelerators are primarily designed for weight-static NNs that involve MM between the learned weight matrix and input tensors and thus are inefficient in supporting attention mechanisms that require dynamic input operands. Here we propose an attention mechanism relying solely on the runtime-programable optical-interference. Through theoretical analyses, numerical simulations and experimental validations, we demonstrate the photonic “all-interference” attention with learning capability equivalent to classical self-attention, and implement the photonic transformer chip (PTC). Evaluation shows that the PTC is promising to exceed 200 pera-operations per second (POPS) with 1POPS/mm<sup>2</sup> computation density and 0.5 POPS/W power efficiency, much better than prior photonic accelerators, and delivers over 200 × energy reduction and 2 to 3 orders of magnitude higher computation capability compared to the electronic counterpart. The photonic transformer with “all-interference” attention proposed in this work highlights the immense potential of photonics to construct its own computing paradigm for general purpose machine learning.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Photonic transformer chip: interference is all you need

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年10月31日

查看全文
  • Ye Tian,
  • Shuiying Xiang,
  • Xingxing Guo,
  • Yahui Zhang,
  • Jiashang Xu,
  • Shangxuan Shi,
  • Haowen Zhao,
  • Yizhi Wang,
  • Xinran Niu,
  • Wenzhuo Liu,
  • Yue Hao

摘要

As the core component of the transformer model, the attention has been proved as all you need in artificial intelligence field in recent years. However, conventional electronic processors are unable to cope with the exponentially increasing hardware costs and energy consumption of the computing-expensive attention. While the photonic neural network (NN) chips provide alternative energy-efficient solutions for accelerating the matrix multiplication (MM), existing photonic accelerators are primarily designed for weight-static NNs that involve MM between the learned weight matrix and input tensors and thus are inefficient in supporting attention mechanisms that require dynamic input operands. Here we propose an attention mechanism relying solely on the runtime-programable optical-interference. Through theoretical analyses, numerical simulations and experimental validations, we demonstrate the photonic “all-interference” attention with learning capability equivalent to classical self-attention, and implement the photonic transformer chip (PTC). Evaluation shows that the PTC is promising to exceed 200 pera-operations per second (POPS) with 1POPS/mm2 computation density and 0.5 POPS/W power efficiency, much better than prior photonic accelerators, and delivers over 200 × energy reduction and 2 to 3 orders of magnitude higher computation capability compared to the electronic counterpart. The photonic transformer with “all-interference” attention proposed in this work highlights the immense potential of photonics to construct its own computing paradigm for general purpose machine learning.