Despite significant recent progress in the field of recommendation systems, modern reinforcement learning (RL) methods still struggle to understand uncommon items and reason user dynamic interests based on their history behaviors. On the one hand, Large Language Models (LLM) have shown superior context understanding, extensive knowledge and impressive reasoning capabilities but are still hindered by the high computational demand and slow inference speed for direct online deployment. To address the issues, this paper introduces LLM-AC, a novel LLM enhanced Actor-Critic framework to improve the RL recommend policy. In detail, LLM-AC fine-tunes LLM with the Low-Rank Adaptation (LoRA) technique for user behavior prediction to better understand context and capture user dynamic interests. Then, the fine-tuned LLM enhances the Actor-Critic by: (i) LLM acts as a user behavior prediction model to enhance value estimation of the critic considering user dynamic interests, and (ii) LLM enhanced critic further guides the actor’s learning to achieve the recommended strategy. Note that only the trained actor is deployed online during the serving phase, which can guarantee the real-time inference speed. Our extensive experiments on public movie and book domains show that LLM-AC significantly improves recommendation performances beyond state-of-the-art baselines. Additionally, the proposed LLM-AC is highly efficient for the inference speed, which is 21 \(\times \) faster than the LLM-based recommender policy (Our code is available at https://anonymous.4open.science/r/LLM-AC-B3B4 ).