The user experience (UX) is a critical factor in the success of e-commerce solutions. A superior UX can set a company apart from its competitors and foster customer loyalty, ultimately leading to increased sales. Therefore, it is essential to maintain and monitor the quality of the UX in an e-commerce solution. As these solutions are dynamic and evolve over time, continuous evaluation is necessary. In this paper, we present a case study that outlines a method for monitoring the UX of an e-commerce solution through a combination of UX surveys and usage tracking. Usage tracking provides detailed information on usage behavior. UX surveys allow to evaluate the satisfaction or dissatisfaction of users with certain aspects of UX. The goal of the presented approach is to combine the strength of both methods. This combination of usage tracking and the UEQ-S questionnaire has been successfully implemented in a live web shop, and we are able to provide initial insights based on several months of data collection.