In the era of widespread intelligent frameworks and models, we are often surrounded by systems that house multiple models for varied task-specific predictions. Given the general expanse of large intelligent frameworks, we explore the use case of a large-scale setup with locally differentiated machine learning models organized in a multi-tier hierarchy. We specifically aim to understand how to support continuous tier-specific inference queries in a hierarchical multi-model setting while optimizing for network costs, storage space and respecting local privacy constraints. This paper explores two intuitive frameworks in this regard and performs a thorough comparative analysis between the frameworks outlining numeric support points for the approaches. The paper also introduces a novel dynamic protocol, termed Meet-in-the-Middle (MITM), that combines the benefits of the previously introduced protocols using a novel meet-in-the-middle, staggered model-caching approach. Thorough testing and analysis of MITM on a distributed agricultural disease-prediction dataset displays the superiority of the novel protocol over the previously introduced frameworks, demonstrating a reduction of up to 80% in real-time communication cost, 80% in memory utilization and 50% in inference latency while maintaining comparable metrics of accuracy at even higher levels of the multi-tier hierarchical framework.