Scholarlink

In the rapidly evolving landscape of autonomous vehicles, fusing data from multimodal sensors such as LiDAR, cameras, radar, and Inertial Measurement Units (IMUs) represents a critical cornerstone for achieving accurate environmental perception and ensuring operational safety. However, real time processing at the edge presents formidable challenges in terms of latency constraints and computational complexity, particularly when handling heterogeneous sensor streams with varying temporal characteristics and data formats. We introduce the Multi-Sensor Fusion Core, a production ready FPGA based sensor fusion framework that achieves remarkable performance metrics: an average latency of 5.51 ms on the KITTI dataset and 13.85 ms on the nuScenes dataset, with a fusion accuracy of 99.3% across diverse operational scenarios. Our system substantially outperforms existing state of the art solutions, including Farag EKF (21.23 ms), BEVFusion (119.2 ms), and MLSF, through a synergy of key technological innovations: linear attention mechanisms replacing computationally intensive Softmax operations for efficient feature fusion, Asymmetric Numeral Systems (ANS) for accelerated entropy decoding with reduced overhead, and Triple Modular Redundancy (TMR) ensuring ASIL-D compliance for safety critical automotive applications. The architecture demonstrates exceptional scalability, deterministic timing characteristics, and remarkable robustness under comprehensive stress testing scenarios, featuring 16 parallel processing cores with an optimized 8 stage pipeline and sophisticated adaptive thermal management capabilities, making it ideally suited for deployment in safety critical autonomous vehicle applications.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-Sensor Fusion Core: A Real Time FPGA Based Sensor Fusion Architecture for Autonomous Vehicles with Ultra Low Latency and High Reliability

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Ngo Duc Anh,
  • Tran Ngoc Thinh,
  • Huynh Phuc Nghi,
  • Long Tan Le

摘要

In the rapidly evolving landscape of autonomous vehicles, fusing data from multimodal sensors such as LiDAR, cameras, radar, and Inertial Measurement Units (IMUs) represents a critical cornerstone for achieving accurate environmental perception and ensuring operational safety. However, real time processing at the edge presents formidable challenges in terms of latency constraints and computational complexity, particularly when handling heterogeneous sensor streams with varying temporal characteristics and data formats. We introduce the Multi-Sensor Fusion Core, a production ready FPGA based sensor fusion framework that achieves remarkable performance metrics: an average latency of 5.51 ms on the KITTI dataset and 13.85 ms on the nuScenes dataset, with a fusion accuracy of 99.3% across diverse operational scenarios. Our system substantially outperforms existing state of the art solutions, including Farag EKF (21.23 ms), BEVFusion (119.2 ms), and MLSF, through a synergy of key technological innovations: linear attention mechanisms replacing computationally intensive Softmax operations for efficient feature fusion, Asymmetric Numeral Systems (ANS) for accelerated entropy decoding with reduced overhead, and Triple Modular Redundancy (TMR) ensuring ASIL-D compliance for safety critical automotive applications. The architecture demonstrates exceptional scalability, deterministic timing characteristics, and remarkable robustness under comprehensive stress testing scenarios, featuring 16 parallel processing cores with an optimized 8 stage pipeline and sophisticated adaptive thermal management capabilities, making it ideally suited for deployment in safety critical autonomous vehicle applications.