Recent advances in information retrieval introduced Large Language Models (LLM) as chatbot assistants. However, LLM’s reliance on implicit knowledge alone is prohibitive for knowledge-intensive tasks. The retrieval-augmented generation (RAG) technique extends the capabilities of LLMs by supplementing contexts to generate contextually relevant responses. RAG relies on a retriever model that selects documents from a knowledge database. In niche industries with limited training data, generic retriever models often underperform, leading to incorrect chatbot responses. This paper aims to improve a drilling text document retriever by adapting a generic dense embedding model. Using a contrastive learning technique, we aim to maximize the similarity scores of relevant items while pushing dissimilar items apart. We train and evaluate the models based on retrieval accuracy using the Norwegian Petroleum Directorate data. Our results show that contrastive learning enabled the adapted model to retrieve 94% more relevant information than its generic version. The adapted model also outperformed the best commercial model by 80% despite having 6x fewer dimensions. Moreover, based on our sensitivity analysis, the different hyperparameter configurations resulted in minimal variations in the adapted model performance. We release our generated datasets for the public to facilitate further research.