The automatic ability to recognize and analyze human behavior from video data has become increasingly important across various fields, including surveillance, healthcare, robotics, and human–computer interaction. Deep learning-based methods have shown exceptional promise in tackling this recognition challenge. A Motion History Mask-based Convolutional Neural Network (MHM-CNN) is proposed in this research work for Human Action Recognition (HAR). The model selectively focuses on active pixels that are involved in the motion, identified by the Motion History Mask (MHM), prioritizing regions of significant motion while effectively disregarding static background pixels. This targeted approach reduces computational costs and enhances recognition accuracy. The proposed MHM-CNN’s performance is evaluated with four publicly available benchmark datasets. The results demonstrate that MHM-CNN consistently outperforms existing models by achieving top accuracies of 99.77, 99.85, 98.67, and 98.83% for the datasets UCF50, UCF101, KTH, and HMDB51, respectively, with minimal computational effort. These findings underscore the potential of MHM-CNN as a more efficient and accurate solution for complex action recognition tasks.