As an important cornerstone of cross-language information processing, bilingual dictionaries play an indispensable role in the fields of cross-language information retrieval and machine translation. Traditional bilingual dictionary construction mainly relies on manual compilation, which is difficult to meet the growing language data and diversified application needs. This study is based on the comparable corpus and CNN, and uses CNN to extract features from the texts in the corpus to capture semantic associations between words. The core result of the research is to construct a two-tower structure model integrating comparable corpus and CNN. The two CNN towers process input text pairs separately without explicitly relying on each other’s sequence information, and extract local semantic features through multi-layer convolution kernel to preserve word order information while reducing redundancy. The experimental results show that this algorithm can identify and align bilingual words more accurately, and provide more reliable results for the extraction of bilingual dictionaries.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Bilingual Dictionary Extraction Algorithm Based on Comparable Corpus and CNN

  • Xiaoling Yu,
  • Xin Liu,
  • Aijun Liu

摘要

As an important cornerstone of cross-language information processing, bilingual dictionaries play an indispensable role in the fields of cross-language information retrieval and machine translation. Traditional bilingual dictionary construction mainly relies on manual compilation, which is difficult to meet the growing language data and diversified application needs. This study is based on the comparable corpus and CNN, and uses CNN to extract features from the texts in the corpus to capture semantic associations between words. The core result of the research is to construct a two-tower structure model integrating comparable corpus and CNN. The two CNN towers process input text pairs separately without explicitly relying on each other’s sequence information, and extract local semantic features through multi-layer convolution kernel to preserve word order information while reducing redundancy. The experimental results show that this algorithm can identify and align bilingual words more accurately, and provide more reliable results for the extraction of bilingual dictionaries.