Swarm intelligence emerges from simple local interactions. We show a formal equivalence between pheromone-mediated aggregation in C. elegans and reinforcement learning: stigmergic signals implement distributed rewards and cross-learning updates. Using a foraging model grounded in empirical data, we reproduce C. elegans patch-selection in static environments. In dynamic settings, persistent pheromones create positive feedback that hinders adaptation by locking swarms to outdated sites. Computational bandit experiments reveal that implementing heterogeneity through a minority of exploratory, pheromone-insensitive agents restores plasticity and enables rapid task switching. Thus, behavioural heterogeneity balances exploration-exploitation and implements swarm-level extinction of obsolete strategies. Our results reinterpret stigmergy as externalised memory for collective credit assignment and link synthetic biology with RL-driven swarm control, pointing to programmable living systems capable of resilient decision-making.