Matrix multiplication is a fundamental operation with significant applications across diverse fields, such as physics, electronics, and artificial intelligence. Traditional implementations of this operation exhibit cubic time complexity, which presents computational challenges, particularly in deep learning scenarios that necessitate large-scale matrix computations. In this study, we introduce ProdNet, a lightweight neural network model designed to autonomously discover efficient matrix multiplication algorithms without the need for extensive computational resources or prior knowledge of existing methods. Our approach seeks to alleviate complexity by minimizing the number of multiplicative operations involved. To achieve this, we utilize a combination of Mean Squared Error (MSE) and a regularization function, targeting weight values to be constrained to 0, 1, or –1. We emphasize the effectiveness of our regularization techniques in accelerating the algorithm discovery process. Furthermore, we demonstrate that ProdNet successfully identifies multiplication algorithms for 2 \(\times \) 2 and 3 \(\times \) 3 matrices, achieving this in a limited number of iterations (starting from 500 iterations for \(2 \times 2\) and 8800 iterations for \(3 \times 3\) )

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ProdNet: A Lightweight Network for Fast Discovery of Matrix Multiplication Algorithms

  • Badia Ouissam Lakas,
  • Chemousse Berdjouh,
  • Khadra Bounane,
  • Mohammed Lamine Kherfi,
  • Oussama Aiadi,
  • Samir Brahim Belhouari

摘要

Matrix multiplication is a fundamental operation with significant applications across diverse fields, such as physics, electronics, and artificial intelligence. Traditional implementations of this operation exhibit cubic time complexity, which presents computational challenges, particularly in deep learning scenarios that necessitate large-scale matrix computations. In this study, we introduce ProdNet, a lightweight neural network model designed to autonomously discover efficient matrix multiplication algorithms without the need for extensive computational resources or prior knowledge of existing methods. Our approach seeks to alleviate complexity by minimizing the number of multiplicative operations involved. To achieve this, we utilize a combination of Mean Squared Error (MSE) and a regularization function, targeting weight values to be constrained to 0, 1, or –1. We emphasize the effectiveness of our regularization techniques in accelerating the algorithm discovery process. Furthermore, we demonstrate that ProdNet successfully identifies multiplication algorithms for 2 \(\times \) 2 and 3 \(\times \) 3 matrices, achieving this in a limited number of iterations (starting from 500 iterations for \(2 \times 2\) and 8800 iterations for \(3 \times 3\) )