Network latency in the IOT system causes significant delays because of the 4 layers of IOT in existing research models in transferring data in record time for real-time monitoring. The problem is the need for an efficient and automated system that can accurately and timely detect and manage sweet potato pests and diseases. This work established a hybridized architecture combining Deep Learning and the Internet of Things (IoT) for managing diseases and pests of sweet potatoes. The Object-Oriented Analysis and Design Methodology (OOADM) was employed in the system's design. The hybridized architecture comprised of the Ardunio ESP32 Processor and a raspberry Pi 4 processor, powered by solar. The existing system had a computational cost of training for 9 h, 30 min, while the proposed system training lasted for 5 h, 30 min, with 4 frames per second of camera capture. This implied that the proposed system saved 4 h out of the Training time and increased the latency rate of the model for remote monitoring.