Construction of the twenty-four histories ancient-modern part-of-speech tagged corpus
摘要
Corpora are crucial in computational linguistics and natural language processing. With the advancement of large language models (LLMs), high-quality human-annotated corpora have become essential for training high-performance general and domain-specific LLMs. This paper presents the construction of a part-of-speech (POS) tagged corpus based on the Twenty-Four Histories and their corresponding modern Chinese translations. First, in the data preprocessing stage, methods such as regular expressions, language models, and manual proofreading were employed to ensure data quality. In addition, task-specific annotation guidelines were established to standardize the POS tagset. Subsequently, the distribution patterns at the lexical level in the constructed corpus were explored from dimensions including word length, word frequency, POS tag distribution, word co-occurrence frequency, POS tag co-occurrence frequency, and word collocation relationships. Finally, we discuss potential applications. This corpus is released to support digital humanities research on ancient Chinese and to facilitate the intelligent processing of classical texts.