Privacy Promise Vs. Tracking Reality in Pay-or-Tracking Walls

European websites increasingly adopt pay-or-tracking walls, sometimes known as “consent or pay models,” “cookie paywalls,” or “pay-or-okay walls.” These walls require users to pay a fee or consent to be tracked in exchange for website access. However, initial evidence suggests that websites might continue to track users even when they pay the fee, constituting user deception. This paper comprehensively assesses whether websites employing pay-or-tracking walls keep their privacy promise to paying users as stated on the pay-or-tracking wall and safeguard their privacy. Data collection and analysis from 341 websites show that while websites reduce tracking for paying users, 32.9% of the websites fail to uphold the privacy promise declared on their pay-or-tracking wall. 80% of these websites could meet their privacy commitments by removing just one or two trackers. Notably, a group of websites offering a joint subscription allowing access to all participating websites better keeps their privacy promises than others, likely due to the implementation of an ongoing control mechanism that regularly detects tracker usage. The results show that implementing tracking-free websites remains challenging and might require continuous efforts.