Comparative Analysis of Large Language Models for Question Answering from Financial Documents

Extracting and analyzing information from financial documents is necessary to understand the economic growth of any business and country. This information is required to make investments, policy formation, and to take other crucial decisions to increase profits. Huge volume of financial documents makes it very difficult and time taking process to extract useful information. Question answering is a powerful way to extract relevant information quickly. Recent research has demonstrated that the large language models (LLMs) give state-of-the-art results for various natural language processing tasks such as question answering, document classification, sentiment analysis, and many more. Extracting relevant details from financial documents is different from getting answers from general document corpus. Mathematical and logical reasoning is also required to retrieve information from financial documents. In this paper we present a comparative analysis of two popular LLMs for question answering from financial documents: OpenAI’s ChatGPT and Meta AI’s LLaMA. While ChatGPT API is propriety in nature, LLaMA’s model weights are available freely for research. The experimental results show that the performance of LLaMA is comparable with ChatGPT for question answering from financial documents.