FPGA-Based Parallel Acceleration of State Space Models for Deep Learning

State Space Models (SSMs) have emerged as a powerful tool in deep learning, offering exceptional capabilities for modeling long-range dependencies, multi-modal fusion, and computationally efficient inference. Despite their theoretical advantages, there is still a research gap in the exploration of their hardware efficiency, especially the performance on dedicated platforms. This paper presents a novel FPGA-based acceleration architecture tailored for SSMs. By leveraging parallel state update to overcome sequential bottlenecks in state updates and employing block ram buffering techniques to minimize external memory access, our architecture achieves significant performance gains. Utilizing Xilinx Zynq UltraScale+ devices, this paper achieve real-time processing with latencies as low as 1 ms. Experimental results also demonstrate a 2.15× improvement in Tera-Operations per Second per Watt (TOPS/W) compared to GPU implementations. These results demonstrate FPGA’s capability to deliver highly efficient, real-time SSMs processing.