Optimization Under Noise
摘要
This chapter addresses the challenge of optimizing noisy objective functions in derivative-free optimization (DFO), a common issue in real-world applications like reinforcement learning. While the Racos algorithm is effective in noise-free environments, it struggles with noisy evaluations. The chapter introduces value suppression, a novel noise-handling mechanism that delays noise mitigation until the best-so-far solution stagnates, reducing computational costs compared to traditional methods like sampling and threshold selection. The mechanism is integrated into the SRacos algorithm, resulting in SSRacos, which is shown to outperform other noise-handling techniques in both synthetic functions and OpenAI Gym tasks. Empirical results demonstrate that value suppression improves optimization efficiency and convergence under noise, making it a promising approach for noisy DFO problems. The chapter concludes with a discussion on the mechanism’s potential applicability in noise-free environments.