Towards a Digital Archivist: Applications of LLMs in Automated Web Archive Description
摘要
Generating high-quality descriptions for web archives remains a time-consuming bottleneck in digital preservation workflows. This paper explores the use of large language models (LLMs) to automate this task, focusing on fine-tuning the Qwen3-8B model on a curated corpus of human-written summaries. The resulting system produces semantically accurate and context-aware descriptions of HTML records extracted from WARC files. We integrate the model into a full-stack pipeline that handles ingestion, parsing, and AI-driven analysis. Performance is evaluated using BERTScore and MoverScore, alongside manual assessments by archivists, librarians, and archival science students. Results show low Composite Edit Rates (CER) and high user satisfaction, validating both the reliability and utility of LLMs for metadata generation. These findings highlight the potential of AI to enhance scalability, consistency, and quality in archival description practices.