VIKA: Vectorized Indispensable Knowledge-Subgraph Augmentation for Large Language Models

Retrieval Augmented Generation is an effective method that integrates language models with external knowledge sources to achieve better performance. However, they often face certain challenges when dealing with extremely complex tasks. For example, the amount of external knowledge might be excessively huge, creating difficulties for language models to handle long contexts. In addition, for questions involving graph data, how to enable the model to understand the graph structure is also a key issue, since the sequence modeling approach of language models differs significantly from the structure of graphs. In this paper, we propose a Vectorized Indispensable Knowledge-subgraph Augmentation (VIKA) method for large language models. We utilize the vectorized graph augmenting method to cover long contexts and integrate graph structure information through an adapter, which helps to maximize the inclusion of valuable knowledge. We add a hallucination detection module before retrieval and a comprehensive filtering method to avoid irrelevant disturbance and promote the quality of knowledge and the accuracy of enhancement. Experiments across three different tasks demonstrate that our method achieves significant improvements compared to other models.