Evaluation of Intrinsic Explainable Reinforcement Learning in Remote Electrical Tilt Optimization

This paper empirically evaluates two intrinsic Explainable Reinforcement Learning (XRL) algorithms on the Remote Electrical Tilt (RET) optimization problem. In RET optimization, where the electrical downtilt of the antennas in a cellular network is controlled to optimize coverage and capacity, explanations are necessary to understand the reasons behind a specific adjustment. First, we formulate the RET problem in the reinforcement learning (RL) framework and describe how we apply Decomposed Reward Deep Q Network (drDQN) and Linear ModelU-Tree (LMUT), which are two state-of-the-art XRL algorithms. Then, we train and test such agents in a realistic simulated network. Our results highlight both advantages and disadvantages of the algorithms. DrDQN provides intuitive contrastive local explanations for the agent’s decisions to adjust the downtilt of an antenna, while achieving the same performance as the original DQN algorithm. LMUT reaches high performance while employing a fully transparent linear model capable of generating both local and global explanations. On the other hand, drDQN adds a constraint on the reward design that might be problematic for the specification of the objective, whereas LMUT could generate misleading global feature importance and needs additional developments to provide more user-interpretable local explanations.