This research proposes a novel Smart Fog Gateway architecture augmented with machine learning (ML) algorithms to significantly reduce latency in fog-based IoT environments. Unlike conventional fog optimization methods, our gateway dynamically offloads and processes tasks at the edge by predicting computational loads using ML classifiers such as Decision Trees, SVM, and Random Forests. The architecture supports real-time data inference while delegating model training to the cloud, optimizing both computation and energy consumption. Experimental evaluations using a Raspberry Pi-based fog node and Google Colab simulations reveal that our approach improves latency by up to 20–25%, with Decision Tree classifiers demonstrating superior performance in resource-constrained edge settings. This system enables intelligent, adaptive, and scalable fog computing suitable for heterogeneous IoT networks.