The quality of healthcare can be measured in many ways, from staffing to survival rates. England’s National Health Service is highly regarded worldwide, but scandals led to increased scrutiny and calls for greater monitoring. Our involvement in two public inquiries led to us develop a national hospital mortality monitoring system using routinely collected inpatient data. Our commercial partners designed the system’s web-based front end to alert hospitals when their death rates were significantly above the national average. We devised the technical details of this system, which included how to define clinically important patient groups, how to adjust for patient risk, and how to limit the false alarm rate when monitoring hundreds of patient groups and hospitals over time. We derived an equation to tailor the alerting threshold to the desired false alarm rate, the size of the hospital, and the death rate for each patient group. Our system immediately flagged the high death rate at Mid Staffordshire NHS Trust. It transformed Dr Foster’s business model, their expansion leading to substantial investment from the Department of Health, and our mixed-methods evaluation showed its positive impact on the NHS and patients. It continues to help many hospitals monitor and improve their performance.