The trade-off between latency and accuracy of Convolutional Neural Networks (CNN) presents challenges in designing CNN-based perception for autonomous systems. Occasionally, the plant requires quick decisions to ensure the safety of the mission, whereas, at other times, it requires confident decisions to ensure the success of the mission. In this work, we propose a situation-aware Model Predictive Control (MPC) to integrate perception and control modules. This framework enables the dynamic adjustment of latency and accuracy parameters by choosing between two distinct CNNs, based on situational demands. This facilitates the attainment of optimal system performance. The proposed approach is evaluated both in simulator and real-world testbed. The results indicate that the i) proposed method surpasses baseline methodologies and incorporates ii) computational optimizations, further enhancing its efficacy.