<p>In crowded environments, abnormal behavior recognition is a significant task for public safety, which requires the development of robust and effective surveillance systems. Conventional approaches face considerable limitations including occlusions, fluctuating crowd densities, and varying environmental conditions. This research proposes the Cross Attentive Spatio-Temporal Aggregation based Threshold Ensemble model to detect abnormal behavior in crowded environments, which rectifies the aforementioned limitations. The data associated with abnormal behaviors are collected from three datasets and the data is in the form of videos, therefore some pre-processing steps are performed to convert the videos into various frames for further analysis. The proposed methodology employs cross attentive spatio-temporal aggregation model for feature extraction, in which cross attentive aggregation aggregates the spatial and temporal information by concentrating significant areas. Additionally, variance-based attention refines abnormal behavior detection by concentrating on rapidly moving areas within video sequences. Afterward, the proposed methodology employs a thresholding-based ensemble machine learning model for the detection and classification of different behaviors in a crowded environment. Further, the tracking rules are used in this research to track human movements and behaviors, particularly identifying the directions of crowds as they navigate crossing trajectories. The proposed model is validated on three datasets and demonstrates superior performance, attaining 98.82% object tracking accuracy and 98.55% precision. The results demonstrate the efficiency of the proposed model in detecting abnormal behavior and also it outperforms baseline models in this field.</p>