Comparison and Hyperparameter Analysis of Four Reinforcement Learning Algorithms in the Lunar Lander Environment

Reinforcement learning trains an intelligent agent to take actions in an environment in order to maximize the cumulative reward. Provided the OpenAI Gym environment LunarLander-v2, both policy-based and value-based methods are implemented to train the lander model in order to achieve an excellent landing, considering factors such as the moving speed, time consumed, and landing position. The performances were compared and further optimized. In this paper, the work compares Deep Q-Network (DQN), Double Deep Q-Network (DDQN), Policy Gradient (PG), and Advantage Actor Critic (A2C) to train the model. By adjusting the values of key hyperparameters, several ways are identified to improve the efficiency and effectiveness of the reinforcement learning process. Ultimately, DQN emerged as the most effective option. This work employed a neural network with 128 replay samples and 256 hidden layer units to the agent and selected a greedy policy with ε-decay set to 0.996 and a discount value of 0.99 in the Q-function to train the agent, which displayed the best training performance.