Recent advancements in object detection and tracking systems have revolutionized industries like surveillance and autonomous vehicles. These systems are vital for real-time object identification and monitoring, aiding tasks from security to inventory management. However, current systems face challenges in accurately handling dynamic environments due to issues like noise reduction and adaptability to changing scenarios, affecting their performance and efficiency. To address these challenges, proposed methodology integrates techniques such Adaptive Guided Multilayer Side Window Box Filter (AGMSWBF) for noise reduction and enhancing visual quality, along with segmentation methods like Hesitant Fuzzy Threshold Linguistic Bi-Objective Clustering (HFTL-BiOC). Furthermore, proposed approach incorporates a Siamese Network based Incremental Spatio-Temporal Learner (SN-ISTL) with coronavirus mask protection optimization for improved object classification. Additionally, utilized the enhanced tracking learning detection for fine tuning neural network performance. The proposed framework demonstrates exceptional performance with high accuracy, precision, and low false acceptance and rejection rates of 99.26%, 96.59%, 0.0912%, and 0.0987%, respectively. By integrating these methods, proposed approach aims to enhance OD and tracking accuracy, adaptability to changing environments, and efficiency in processing spatio-temporal data streams, thereby overcoming the limitations of existing methodologies.