Recommender Systems (RS) have demonstrated significant potential to address information overload by offering tailored suggestions across diverse sectors. While recent advancements have explored the integration of Large Language Models (LLMs) into RS to improve recommendation quality in various domains, their potential for enhancing Personal Information Assistance (PIA) remains relatively underexplored. In this study, we leverage the strength of LLMs to improve Entity Recommendation, one of the key applications of RS in PIA. Our approach focuses on refining recommendations through the incorporation of explicit user feedback using the Mistral 7B model. We evaluate our method using a benchmark built upon RLKWiC, a dataset of Real-Life Knowledge Work in Context. Results demonstrate that our LLM-based approach outperforms the previously proposed semantic-based method in recommending contextually relevant entities.