<p>Constrained synthesizability is an unaddressed challenge in generative molecular design: in particular, designing molecules satisfying multi-parameter optimization objectives, while simultaneously being synthesizable and enforcing the presence of specific building blocks in the synthesis. This is practically important for molecule re-purposing, sustainability and efficiency. Here we propose the Tanimoto Group Overlap (TANGO) reward function, which uses chemistry principles to transform a binary reward function into a continuous reward function. TANGO can augment molecular generative models to directly optimize for constrained synthesizability using reinforcement learning (RL). Our framework is general and addresses starting-material, intermediate and divergent-synthesis constraints. Contrary to many existing works in the field, we show that incentivizing a general-purpose model with RL is a productive approach to navigating challenging synthesizability optimization scenarios.</p>