<p>Heart failure with reduced ejection fraction (HFrEF) presents a significant diagnostic challenge, particularly in advanced stages. Deep learning (DL) models offer promising potential for automated HFrEF detection, but their effectiveness is often limited in small and imbalanced clinical datasets. In this study, we propose a novel approach to enhance HFrEF detection from echocardiographic videos. We adapt the TimeSformer architecture—a Transformer-based model optimized for spatiotemporal feature extraction in video data—and apply it for the first time in echocardiography. In addition, we introduce a domain-informed left ventricle (LV) masking method using image segmentation to guide model attention toward diagnostically critical regions, boosting overall performance. Our methodology is evaluated both directly on a large-scale benchmark dataset and on a smaller, specialized clinical dataset from our cardiology department after fine-tuning. Experimental results show that the proposed framework yields substantial performance gains: on the benchmark dataset, we observe a 3% improvement in both model accuracy and area under the curve (AUC), while on the specialized dataset, improvements reach 7% in accuracy and 30% in AUC values. Moreover, TimeSformer consistently outperforms conventional approaches after the implementation of fine-tuning and LV masking. These findings highlight a practical and generalizable strategy for improving automated HFrEF diagnosis, with strong implications for clinical decision support in data-scarce healthcare settings.</p>