This paper addresses decision-making problems where the objective is to maximize the best outcome along a trajectory in a Markov Decision Process (MDP), rather than its cumulative reward. Such extremal objectives naturally arise in risk-sensitive applications, including cybersecurity and resilience planning, but fall outside the scope of classical MDP theory due to their non-Markovian nature. We propose a principled transformation that augments the MDP state space with a deterministic memory variable tracking the maximal reward, yielding an equivalent total-reward MDP solvable by dynamic programming. This construction enables the extraction of Markovian policies through a provision function tailored to each initial state. We evaluate our framework on a malware containment problem, showing that the memory-based baseline policy significantly outperforms a greedy myopic strategy across all configurations. Our results demonstrate improved worst-case cost and containment time, highlighting the utility of memory-augmented planning for extremal performance control in stochastic systems.