Metamorphic Testing for Recommender Systems

Recommender systems are commonly based on a multi-armed bandit model. This model should be carefully tested because it affects the users, but it is technically complicated because of the test oracle problem and the stochastical nature of multi-armed bandit algorithms. Metamorphic testing is a testing method for problems without test oracles. In this paper, we propose a novel approach that applies metamorphic testing to the verification of the requirements for stochastic models. We propose a stochastic metamorphic relation (SMR) which is a composition of a sampling procedure and a determination function. We propose several relations for multi-armed bandit models and algorithms. Then, we implement those relations and test algorithms. Our experiment demonstrates that the proposed method can identify errors and that stochasticity of relations is essential.