Rethinking the role of attention mechanism: a causality perspective

As the core technology of Transformers, the attention mechanism is almost indispensable. However, many experimental findings show that the models developed based on the attention mechanism are not as perfect as imagined, and there are pitfalls in their ability to capture effective information, especially in some multi-modal tasks. In this paper, we continue to delve into this issue and try to uncover the mysterious nature of the attention mechanism through powerful explainable causal inference techniques. At the theoretical level, we rigorously characterize the capacity bottleneck of the attention mechanism in multi-modal tasks and demonstrate the shortcomings of the attention model in its ability to weed out invalid features. Further, we obtain results consistent with the theoretical analysis in the experimental session. In particular, the model optimized under the guidance of our theoretical analysis achieves superiority over state-of-the-art methods in visual question-answering tasks. Excitingly, we find that the attention mechanism’s defects can be repaired, and the repair method has strong generalization properties. This distinct advantage will provide a clear interpretable optimization technique for the attention-based framework.