Detecting and correcting spelling errors in Chinese is a critical challenge in natural language processing, particularly for second language acquisition and assessment. Numerous methodologies have been advanced for this task; however, they typically rely heavily on expert knowledge, such as predefined confusion sets and grammatical rules. This reliance can be problematic in scenarios where expert insights are insufficient. To address this limitation, we introduce a novel end-to-end neural network model that autonomously detects and corrects Chinese spelling errors without depending on expert knowledge or traditional rules. Our model integrates an encoder with a unique gating mechanism, employing convolutional neural networks, and a decoder that utilizes a bi-directional recurrent network. Additionally, we develop a transfer learning strategy to enhance our model’s performance in data-scarce environments. Experimental results demonstrate that our approach significantly surpasses existing baselines in detecting and correcting Chinese spelling errors.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An End-to-End Method for Chinese Spelling Error Detection and Correction

  • Shuangyin Li,
  • Jinbin Zhang,
  • Yuncheng Jiang

摘要

Detecting and correcting spelling errors in Chinese is a critical challenge in natural language processing, particularly for second language acquisition and assessment. Numerous methodologies have been advanced for this task; however, they typically rely heavily on expert knowledge, such as predefined confusion sets and grammatical rules. This reliance can be problematic in scenarios where expert insights are insufficient. To address this limitation, we introduce a novel end-to-end neural network model that autonomously detects and corrects Chinese spelling errors without depending on expert knowledge or traditional rules. Our model integrates an encoder with a unique gating mechanism, employing convolutional neural networks, and a decoder that utilizes a bi-directional recurrent network. Additionally, we develop a transfer learning strategy to enhance our model’s performance in data-scarce environments. Experimental results demonstrate that our approach significantly surpasses existing baselines in detecting and correcting Chinese spelling errors.