An incongruity-aware hybrid quantum neural network for multimodal sarcasm detection
摘要
Accurate sarcasm detection is crucial for understanding social media sentiment but remains challenging due to multimodal incongruities. Current models primarily rely on unimodal sentiment extraction, failing to capture cross-modal discrepancies. This limitation causes information loss and restricts recognition of latent sentiment dynamics, areas where classical neural networks underperform. To address this issue, we propose the incongruity-aware quantum neural network (IAQNN), a novel fusion framework that explicitly identifies and learns sentimental and factual incongruities across modalities. IAQNN integrates: (1) an attention module highlighting objective details for discrepancy detection, (2) a knowledge-enhanced sentiment embedding layer capturing granular, non-binary sentiment representations, and (3) quantum neural layers modeling sarcasm’s inherent ambiguity. Evaluations on benchmark datasets demonstrate IAQNN’s consistent superiority over state-of-the-art methods in real-world multimodal sarcasm detection.