Severe traumatic brain injury (sTBI) necessitates precise cerebral perfusion monitoring to guide treatment decisions and improve patient outcomes. While ultrasound localization microscopy (ULM) offers unprecedented super-resolution microvascular imaging capabilities, its clinical translation remains constrained by requirements for ultra-high frame rate equipment and extensive data processing times. This study presents a Hierarchical Temporal Dynamics Learning Network (HiT-ULM) that achieves end-to-end super-resolution microvascular imaging using clinically accessible low frame rate contrast-enhanced ultrasound (CEUS) data. Our approach integrates hierarchical temporal learning and spatial attention mechanisms within a unified encoder-decoder framework, effectively capturing long-range spatiotemporal correlations in microbubble dynamics while eliminating traditional localization and tracking procedures. Experimental validation on clinical sTBI patient datasets and preclinical pig brain data demonstrates superior performance across multiple evaluation metrics. On sTBI datasets, HiT-ULM achieved 7.2% enhancement in structural preservation, and substantial improvements in spatial metrics with 12.3% increase in Jaccard index and 8.3% improvement in Dice coefficient compared to existing methods. By improving the efficiency of data processing and enhancing the clinical practicality of super-resolution microvascular imaging, HiT-ULM offers promise for advancing the clinical management of sTBI and improving patient outcomes.