The concept of using intelligent visual recognition to detect abnormal human behavior has significantly improved the ideals of surveillance systems, situational awareness, national security, and intelligent environments. This progress comes with challenges arising from the diversity of anomalous activities, including the definition of anomalies, the representation of their characteristics, their practical applications, and the datasets used. In recent decades, rapid technological advances have led to the widespread use of large-scale surveillance systems in public places such as shopping malls, hospitals, airports, train stations, bus stops, and streets. These systems have great potential in managing situations, improving collective security, and countering threats. However, the proliferation of multi-camera surveillance systems that capture different angles and scenes has made manual monitoring more complex. The available literature presents practical designs for specific situations, including scenarios such as fall detection, ambient assisted living (AAL), homeland security, surveillance, and crowd analysis. These designs use a variety of data sources, including: RGB, depth, and skeleton information. Additionally, recently introduced datasets in the field of human anomalous activity detection (AbHAR) are highlighted in the literature. These datasets were carefully selected by researchers with the aim of improving method validation and complexity of the research area.