Scholarlink

<p>Addressing the practical challenges of university personnel archives, including image-centric storage, heterogeneous page layouts, weak semantic retrieval, and heavy manual cataloguing workload, this study proposes a multimodal fusion retrieval framework based on the reinforced perceptual classification algorithm MRDPA. The proposed framework aims to support the full archive service pipeline of automatic classification, structured information extraction, index construction, and unified retrieval. In MRDPA, archive images are first encoded using a grid-based visual representation, where ResNeXt-FPN scales each sample to 224 × 224 and generates 7 × 7 sub-feature maps through average pooling. Textual information is obtained using OCR and BERT-based semantic encoding, while layout information is represented by spatial bounding-box embeddings. A spatially aware Transformer is then used to fuse visual, textual, and layout features. To enhance fine-grained visual cues in tables, forms, seals, and sparse-text pages, a multi-scale shallow visual enhancement module is further introduced by injecting complementary visual prompts into shallow Transformer layers. Experimental results show that the proposed method achieves an overall classification accuracy of 97.33% for seven categories of personnel archive images, with macro-averaged precision, recall, and F1-score reaching 96.82%, 96.39%, and 96.58%, respectively. The precision of both Resume and Party/League Membership categories reaches 100%, and all categories maintain F1-scores above 91%. In the information extraction task, MRDPA shows more balanced precision and recall than the comparison models. Moreover, the triplet extraction F1-score increases from 0.906 to 0.923 when the sample size increases from 50 to 100, but only increases to 0.931 when the sample size increases from 100 to 200, indicating diminishing marginal returns and providing guidance for annotation cost control. The results demonstrate that MRDPA can improve archive classification and retrieval accuracy while providing a reusable technical path for intelligent, traceable, and service-oriented university personnel archive management.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimizing university personnel archives via a multimodal fusion retrieval system with reinforced perceptual classification

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月3日

查看全文
  • Ying Zheng,
  • Tang Li,
  • Bin Luo

摘要

Addressing the practical challenges of university personnel archives, including image-centric storage, heterogeneous page layouts, weak semantic retrieval, and heavy manual cataloguing workload, this study proposes a multimodal fusion retrieval framework based on the reinforced perceptual classification algorithm MRDPA. The proposed framework aims to support the full archive service pipeline of automatic classification, structured information extraction, index construction, and unified retrieval. In MRDPA, archive images are first encoded using a grid-based visual representation, where ResNeXt-FPN scales each sample to 224 × 224 and generates 7 × 7 sub-feature maps through average pooling. Textual information is obtained using OCR and BERT-based semantic encoding, while layout information is represented by spatial bounding-box embeddings. A spatially aware Transformer is then used to fuse visual, textual, and layout features. To enhance fine-grained visual cues in tables, forms, seals, and sparse-text pages, a multi-scale shallow visual enhancement module is further introduced by injecting complementary visual prompts into shallow Transformer layers. Experimental results show that the proposed method achieves an overall classification accuracy of 97.33% for seven categories of personnel archive images, with macro-averaged precision, recall, and F1-score reaching 96.82%, 96.39%, and 96.58%, respectively. The precision of both Resume and Party/League Membership categories reaches 100%, and all categories maintain F1-scores above 91%. In the information extraction task, MRDPA shows more balanced precision and recall than the comparison models. Moreover, the triplet extraction F1-score increases from 0.906 to 0.923 when the sample size increases from 50 to 100, but only increases to 0.931 when the sample size increases from 100 to 200, indicating diminishing marginal returns and providing guidance for annotation cost control. The results demonstrate that MRDPA can improve archive classification and retrieval accuracy while providing a reusable technical path for intelligent, traceable, and service-oriented university personnel archive management.