Current approaches to dependable AI inference in EdgeAI systems are problematic. The massive parallelization implied by GPUs and NPUs does not lend itself to traditional, that is redundant, forms of execution for integrity checking. State of the art inference-runtimes use a data streaming approach to accelerate inference under conditions of low available memory and, implemented in a lock-free manner, exhibit high orchestration efficiency in the face of low computational power. Under these conditions the actual mathematical operations act as a bottleneck. In low-power operations, imperative for the target domain of Space and indeed any battery-operated system, increasing the clock rate simply isn’t an option. Radiation-hardened circuitry, despite its high cost, may offer an alternative. We propose the application of the High-Performance Data Processor (HPDP) as a dedicated mathematical backend integrated in the data streaming pipeline of the Klepsydra AI-inference orchestration framework. Our results, comparing the performance of the HPDP against well-known radiation-hardened CPUs, confirm the validity of the approach. Given the architectural features of the HPDP and lessons-learned we then propose an FPGA architecture for which first experimental results indicate comparable performance estimates.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dependable AI Inference - A Work-in-Progress on CPU, Co-processor and FPGA Approaches

  • Hans Dermot Doran,
  • Carlos Rafael Tordoya Taquichiri,
  • Pablo Ghiglino,
  • Mandar Harshe

摘要

Current approaches to dependable AI inference in EdgeAI systems are problematic. The massive parallelization implied by GPUs and NPUs does not lend itself to traditional, that is redundant, forms of execution for integrity checking. State of the art inference-runtimes use a data streaming approach to accelerate inference under conditions of low available memory and, implemented in a lock-free manner, exhibit high orchestration efficiency in the face of low computational power. Under these conditions the actual mathematical operations act as a bottleneck. In low-power operations, imperative for the target domain of Space and indeed any battery-operated system, increasing the clock rate simply isn’t an option. Radiation-hardened circuitry, despite its high cost, may offer an alternative. We propose the application of the High-Performance Data Processor (HPDP) as a dedicated mathematical backend integrated in the data streaming pipeline of the Klepsydra AI-inference orchestration framework. Our results, comparing the performance of the HPDP against well-known radiation-hardened CPUs, confirm the validity of the approach. Given the architectural features of the HPDP and lessons-learned we then propose an FPGA architecture for which first experimental results indicate comparable performance estimates.