Existing video-text retrieval (VTR) methods suffer from two primary limitations: they typically extract features from each modality in isolation and struggle to effectively align information across different granularity levels. These issues neglect potential contextual cues between the two modalities and can lead to a trade-off between capturing global context and fine-grained details. To address these challenges, we introduce the Hierarchical Cross-Modality Interaction (HCMI) framework, a novel approach that performs bidirectional, multi-level feature interaction during training. Our framework systematically aligns features at patch-word, frame-sentence, and global video-sentence levels, leveraging a dual attentive module for deep feature enhancement and a cross-modality adaptor for efficient feature distillation. Crucially, HCMI requires only a single modality’s input during inference, making it highly practical for large-scale retrieval tasks. Our method achieves new state-of-the-art results on several challenging benchmarks, including MSR-VTT, DiDeMo, VATEX, MSVD and Activity-Net, demonstrating its significant advantages.