The escalating demand for energy efficiency and sustainability underscores the pressing need for advanced technologies to optimize energy consumption across diverse domains. Energy wastage has emerged as a critical challenge, contributing to the inability to fulfill the surging consumer need for electrical energy. The paper introduces a cutting-edge approach, proposing a system that leverages advanced object detection techniques to monitor and dynamically control energy consumption in real-time. The system employs a distributed architecture integrating edge devices and computer vision to enhance energy management efficiency. Smart computer vision devices deployed at the edge capture real-time environmental data, utilizing instance and semantic segmentation algorithms for zone segmentation and accurate identification and counting of individuals within these zones. The counted persons determine threshold values, governing the operation of electrical appliances via a relay mechanism. This integration of object detection and smart IoT edge devices facilitates real-time energy monitoring, enabling users to take proactive measures to reduce electricity consumption based on the identified number of persons. When compared with the existing method of Faster- RCNN, the custom YOLOv8 model achieves an accuracy of 92.3%, with a difference of 12.3% and a fast detection rate of 0.92ms. Furthermore, the system provides precise energy forecasting, facilitating proactive load management and optimal resource allocation. This paper delves into the confluence of YOLO-based Region of Interest (ROI) Object Detection and Quadrant Triggering for a Smart Energy Management System, presenting a cutting-edge approach to cut costs, conserve electricity, and foster environmental sustainability.