Open-Vocabulary Human-Object Interaction (OV-HOI) detection aims to overcome the limitations of traditional predefined categories, enabling the understanding of interactions involving unseen actions or objects. While CLIP-based methods have achieved progress, existing approaches confront two key challenges: over-reliance on the final-layer visual features of the CLIP visual encoder leads to insufficient utilization of crucial object details in intermediate layers, and the lack of focus on key interaction regions in HOI decoder outputs reduces the recognition accuracy of unseen classes. To address this, we introduce HAHNet, incorporating the novel Text-Guided Multi-Path Attention Fusion (TG-MPAF) module, which adaptively integrates multi-level visual features via semantic gating and dynamic hierarchical weighting. Meanwhile, the Interaction Semantic Enhancement Attention (ISEAttention) module is designed to automatically locate critical interaction regions using learnable weights and enhance feature representations through a residual mechanism. Experimental results on the HICO-DET and SWIG-HOI benchmark datasets demonstrate that HAHNet achieves competitive performance in OV-HOI detection.