Scholarlink

General matrix multiplication (GEMM) operations are the fundamental building blocks of computational domains including artificial intelligence (AI). As GPU architectures evolve and high-performance AI becomes increasingly important, optimizing GEMM performance becomes a fundamental problem that needs to be addressed. This paper introduces Stream-K++, an enhancement to the promising Stream-K GEMM scheduling algorithm for workload balancing. We expand Stream-K’s scheduling policies from three to seven and implement an efficient solution selection mechanism using Bloom filters. Our approach rapidly eliminates up to \(\sim \) 95.8% of unsuitable configurations while maintaining a 100% true-negative rate. Implemented using the AMD Composable Kernel library and evaluated on AMD Instinct™MI250X GPUs, Stream-K++ demonstrates significant performance gains (up to \(\sim \) 43%) in select scenarios. It remains competitive (within 20% of optimal) for \(\sim \) 60-97.6% of problem sizes. Our flexible framework, implemented in the Open-sieve C++ library, allows for easy adaptation to new problem sizes, scheduling policies, or additional tuning parameters, paving the way for future optimizations in GPU-based GEMM operations.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Stream-K++: Adaptive GPU GEMM Kernel Selection and Scheduling for AI Using Bloom Filters

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Harisankar Sadasivan,
  • Muhammed Emin Ozturk,
  • Muhammad Osama,
  • Chris Millette,
  • Astha Rai,
  • Maksim Podkorytov,
  • John Afaganis,
  • Carlus Huang,
  • Jing Zhang,
  • Jun Liu

摘要

General matrix multiplication (GEMM) operations are the fundamental building blocks of computational domains including artificial intelligence (AI). As GPU architectures evolve and high-performance AI becomes increasingly important, optimizing GEMM performance becomes a fundamental problem that needs to be addressed. This paper introduces Stream-K++, an enhancement to the promising Stream-K GEMM scheduling algorithm for workload balancing. We expand Stream-K’s scheduling policies from three to seven and implement an efficient solution selection mechanism using Bloom filters. Our approach rapidly eliminates up to \(\sim \) 95.8% of unsuitable configurations while maintaining a 100% true-negative rate. Implemented using the AMD Composable Kernel library and evaluated on AMD Instinct™MI250X GPUs, Stream-K++ demonstrates significant performance gains (up to \(\sim \) 43%) in select scenarios. It remains competitive (within 20% of optimal) for \(\sim \) 60-97.6% of problem sizes. Our flexible framework, implemented in the Open-sieve C++ library, allows for easy adaptation to new problem sizes, scheduling policies, or additional tuning parameters, paving the way for future optimizations in GPU-based GEMM operations.