Adaptive Dynamic Inference Framework Using Multi-Route Neural Networks Under Constraints
摘要
Neural networks have been extensively applied across various domains, where they perform tasks periodically after they are deployed. However, the execution of these tasks often faces changing constraints, such as processing latency and energy consumption. Existing static models, which maintain fixed accuracy and cost, are hard to satisfy these dynamic conditions. To overcome this limitation, we propose a dynamic inference framework based on a dual-degree-of-freedom, multi-route neural network. In this framework, we first design and train the dual-degree-of-freedom multi-route neural network such that there are multiple inference paths with different computational complexity in a single model. Subsequently, we introduce a heuristic routing selection method, which dynamically selects the most appropriate inference routing based on real-time constraints. This approach enhances task accuracy compared to static models and adapting to changing conditions. Experimental results on object detection tasks validate the effectiveness of the proposed dynamic inference framework, demonstrating that our method offers a novel inference strategy for deep neural networks in constrained application scenarios.