Question Answering System to Answer Questions About Technical Documentation

This article ventures into the realm of specialized AI systems for question answering, with a specific focus on programming languages, using Rust as the case study. Our research harnesses the capabilities of BERT, a leading model in natural language processing, to explore its effectiveness in interpreting and responding to complex, domain-specific queries. We have developed a novel dataset, derived from Rust’s detailed documentation, which surpasses the usual input size for language models. This dataset serves as a foundation for evaluating BERT’s performance in a domain-specific context, providing a new resource for testing question-answering systems and shedding light on their strengths and limitations in processing specialized technical information. In this paper, we proposed a solution based on retrieval-reader architecture, the fine-tuned RoBERTa model with the usage of the mentioned dataset, and conducted typical tests for said problem. It is shown, that domain-specific question-answering remains a challenging problem.