This study presents MeCog (Memory-Enhanced Cognitive planning), a framework addressing long-term memory challenges in LLM-based decision-making agents. We use a smaller language model (BERT) as long-term memory within an Actor-Critic reinforcement learning architecture, while short-term memory stores game-state information. Experiments on four TextWorld game types show MeCog outperforms existing methods, achieving 12% higher scores and 16% fewer steps to completion. Ablation studies confirm both memory components’ contributions, with long-term memory particularly impacting tasks requiring strategic planning. Our method enhances LLM capabilities without compromising general abilities, maintaining efficiency through lightweight memory architecture.