Cross-modal hashing aims to enable efficient retrieval across heterogeneous modalities by mapping multimodal data into a shared hash space. Although recent CLIP-based cross-modal hashing methods have greatly improved semantic alignment, challenges remain in fully capturing fine-grained cross-modal interactions and in optimizing hash codes for high retrieval accuracy. To address these issues, we propose DFALH (Dual Fusion with Auxiliary Loss Hashing), a novel end-to-end cross-modal hashing framework that enhances both feature fusion and optimization. Specifically, we introduce a dual-stream bilinear fusion architecture to model fine-grained interactions between image and text embeddings, and design an Auxiliary Loss (AUL) to directly supervise the fused feature space. Through joint optimization with a Supervised Semantic Matching Information (SSMI) loss, DFALH effectively learns highly discriminative and semantically consistent hash codes. Extensive experiments on three benchmark datasets demonstrate that DFALH significantly outperforms state-of-the-art baselines across multiple metrics, achieving superior retrieval performance and faster convergence. The results validate the effectiveness of the proposed architecture and optimization strategy for cross-modal retrieval.