Prompting and Consistency Learning Strategies for Multimodal Grammatical Error Correction in Low Error Density Domains
摘要
Grammatical Error Correction (GEC), which aims to correct errors in texts, is a crucial task in natural language processing. While multimodal GEC methods have improved correction performance by utilizing speech features, their performance diminishes in scenarios with low error density. To address this limitation, we propose an attention-aware prompting strategy that leverages linguistic knowledge embedded in the pre-trained decoder to produce more accurate corrections. This approach incorporates continuous prompts into the decoder through an attention mechanism to maximize the exploitation of inherent knowledge. Furthermore, a consistency learning strategy is employed to mitigate noise in the source speech features by minimizing the difference between the source and target speech features. The resulting speech features are aligned and fused with text features. Experimental results on the benchmark datasets demonstrate that our method significantly outperforms strong baselines in terms of the F0.5 evaluation metric.