Scholarlink

Quantization in neural networks typically requires early commitment to a fixed numerical format—either fixed-point or floating-point—and a limited set of word sizes supported by the target hardware. This rigid choice often compromises either model accuracy or hardware efficiency. While FPGAs offer the flexibility to tailor precision per layer, hardware-aware differentiable architecture search (HW-DNAS) methods that explore this space suffer from high computational overhead due to layer duplication. We propose a single-path, quantization-aware training framework integrated with a novel flexible numerical format. This approach allows each layer to learn, during training, whether to use fixed- or floating-point arithmetic and to determine the optimal bit-widths for exponent and mantissa, without duplicating layers. A multi-objective optimization strategy balances accuracy with hardware efficiency throughout training. Our method achieves up to 10X reductions in average bit-width on MNIST, CIFAR-10, and CIFAR-100, with \(\le \) 1% loss in accuracy. By deferring numerical format selection to training time and eliminating HW-DNAS-style overhead, we enable resource-efficient, precision-adaptive deep learning suitable for deployment on reconfigurable hardware.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Single-Path Precision: Differentiable Bit-Width & Numeric-Format Learning for FPGA-Efficient Neural Networks

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Emanuel Trabes,
  • Kawthar Dellel,
  • Carlos Valderrama,
  • Abdelmalik Taleb-Ahmed

摘要

Quantization in neural networks typically requires early commitment to a fixed numerical format—either fixed-point or floating-point—and a limited set of word sizes supported by the target hardware. This rigid choice often compromises either model accuracy or hardware efficiency. While FPGAs offer the flexibility to tailor precision per layer, hardware-aware differentiable architecture search (HW-DNAS) methods that explore this space suffer from high computational overhead due to layer duplication. We propose a single-path, quantization-aware training framework integrated with a novel flexible numerical format. This approach allows each layer to learn, during training, whether to use fixed- or floating-point arithmetic and to determine the optimal bit-widths for exponent and mantissa, without duplicating layers. A multi-objective optimization strategy balances accuracy with hardware efficiency throughout training. Our method achieves up to 10X reductions in average bit-width on MNIST, CIFAR-10, and CIFAR-100, with \(\le \) 1% loss in accuracy. By deferring numerical format selection to training time and eliminating HW-DNAS-style overhead, we enable resource-efficient, precision-adaptive deep learning suitable for deployment on reconfigurable hardware.