The deployment of neural network models on mobile devices with heterogeneous computing resources presents significant challenges. Primarily, the limited computational capacity of mobile devices constrains efficient neural network execution. Additionally, communication overhead between diverse computing elements considerably affects inference latency. We propose MBHNet, an end-to-end heterogeneous inference system that accelerates model execution through a lightweight multi-branch architecture and efficient scheduling across heterogeneous computing devices. This design enables low-latency inference while preserving model accuracy by jointly optimizing model structure and hardware utilization. According to the experimental results, MBHNet reduces FLOPs by 46% to 64% and parameters by 50% to 68%, while maintaining accuracy. It achieves a 60% latency reduction compared to CPU inference, and 35% compared to GPU inference. Compared to NN-Stretch, MBHNet reduces latency by up to 36.5%. Power consumption is 60% lower than CPU and 45% lower than GPU, with energy consumption reduced by 90% and 70%, respectively..