This paper presents Wikipedia-Savvy-RAG, a lightweight retrieval-augmented generation (RAG) system designed for answering STEM questions. Utilizing a small pretrained LLM and a vectorized knowledge base from English Wikipedia, the system is optimized for deployment on consumer-grade hardware. I investigate the impact of retrieval on small LLMs and the efficacy of a simplified fine-tuning approach. Our benchmarks demonstrate that retrieval significantly enhances performance, though fine-tuning offers limited improvement. Results show that concatenating retrieved passages is more effective than averaging probabilities and that k-shot prompting can negatively affect performance. The system’s capabilities are showcased through a Streamlit web application, including experimental Arxiv paper querying features.