Learning-Based Cross-Modal Fusion Weight Model: A Lightweight Multimodal Fusion Method Based on Adaptive Weight Learning

Multimodal intelligent systems require effective fusion of heterogeneous modalities (text, image, speech). Conventional approaches suffer from static weights, high complexity, and limited ability to adjust modality importance. We propose a lightweight cross-modal fusion model with adaptive weight learning. A three-layer design—Feature Engineering, Lightweight Fusion Learner, and Quality-aware Output Layer—maps raw inputs to a unified feature space and predicts dynamic weights. Feature Engineering normalizes length, encodes modality type and time, and extracts auxiliary cues to form a 34-dim vector projected to 64-dim. The Fusion Learner employs multi-head attention and sparse gating for context-aware weighting, with temperature calibration to curb overconfidence. The Output layer performs weight normalization and weighted fusion, yielding quality score, confidence, and uncertainty. Experiments show strong efficiency and accuracy: ~ 18K parameters, < 1ms latency, mean quality 0.8089, confidence 0.8704, and peak throughput 723,545 samples/s. The system delivers high fusion quality and computational efficiency, and is effective for dialogue, content understanding, and recommendation.