Background <p>Automated breast lesion differentiation in ultrasound (US) imaging has advanced through deep learning (DL) techniques. However, existing 2D approaches predominantly rely on individual static images, overlooking valuable temporal information between frames, which limits their performance. While 3D models can utilize this temporal information, their high computational requirements make them impractical in resource-constrained settings.</p> Methods <p>This study introduces an improved framework that combines spatial and temporal lesion features by incorporating consecutive frames from US videos. Unlike existing 2D models, our framework utilizes a multi-channel input strategy to effectively learn lesion characteristics across frames, avoiding computational burden of 3D models. It offers potential use in resource-limited settings and real-time environments. The framework’s effectiveness has been validated on multicenter data from two different regions.</p> Results <p>Extensive experiments demonstrate that the proposed multi-channel input technique significantly outperforms single-image approaches. Across five distinct DL backbone models, the framework consistently achieved higher precision, recall, and AUC values, highlighting the positive impact of temporal information on classification accuracy. Specifically, the multi-channel approach yielded improvements of up to 8.6% in AUC, 9.86% in precision, and 23.68% in recall compared to single-image inputs. These results highlight the potential of the proposed multi-channel framework as an effective and practical solution for breast lesion differentiation.</p> Conclusions <p>By effectively utilizing temporal information without additional computational burden, our proposed approach serves as a computationally efficient alternative to 3D models, advancing DL-based breast US analysis. It shows promising potential for delivering more accurate and accessible diagnostic solutions, making it highly applicable for clinical practice.</p>