This research presents a comparative analysis of Large Language Models (LLMs) to test their ability to understand and analyze documents mainly PDFs. The base concept behind this test is Retrieval Augmented Generation (RAG), which is a method to combine existing data sources with the generative capabilities of an LLM. The paper evaluates open-source models such as Llama 3.2, Gemma, Qwen and Mistral. Various parameters are used to evaluate the model such as speed, accuracy, relevance, grammatical correctness and answer formatting. The evaluation takes into account not just the answers based on the context provided but also answers based on the models’ pre-trained knowledge. The analysis also studies deployment considerations, including resource requirements, scalability, and data privacy. A scoring system is used to conduct the evaluation of the results generated to make the results more objective and minimize any bias or subjectivity. This highlights the growing potential of open-source LLMs for democratizing document analysis while identifying areas for future development and optimization.