Offline Reinforcement Learning: A New Challenge for Symbolic Regression?

Offline reinforcement learningreinforcement learningprovides an approach to addressing the sample efficiency problem in reinforcement learningreinforcement learning. For example, a behaviour policypolicycould already have provided data for some costly (target) task. Additionally, reinforcement learningreinforcement learningtasks that are described in terms of real-valued states and actions represent a particularly challenging scenario. With this in mind, we consider several offline reinforcement learningreinforcement learningproblems as a new challenge for symbolic regressionsymbolic regression. Our motivation being that genetic programming has had a long history of solving regression problems symbolically, i.e. interpretable solutions. Benchmarking two state-of-the-art symbolic regressionsymbolic regressionmethods on the HalfCheetah, Hopper and Walker2d locomotion tasks using offline data sourced from a deep reinforcement learningreinforcement learningpolicypolicyindicates that the resulting solutions appear to be limited by distributional shift (an incremental compounding of errors). Conversely, both a multi-layer perceptronperceptronand XGBoost are able to discover policies that replicate the performance of the original behavioural policy. The benchmarking datasets employed are publicly available for furthering research in this area.