A critical challenge in healthcare systems in low- and middle-income countries is the efficient and equitable allocation of scarce resources, particularly essential medicines1. This problem is complicated by limited high-quality data, which restricts the applicability of traditional data-driven techniques2–5. Here we propose a novel decision-aware machine learning framework for the allocation of essential medicines, which additionally leverages multi-task learning to ensure sample efficiency and catalytic priors to ensure equitable allocation. In collaboration with the national government of Sierra Leone, we performed a staggered, nationwide deployment of our system as a decision support tool. Our econometric evaluation finds an estimated 19% increase in consumption of allocated products in treated districts, demonstrating its efficacy at improving access to essential medicines. Our tool was subsequently scaled nationwide, covering an estimated two million women and children under 5 years of age. Our work demonstrates how machine learning methods can improve efficiency at very low cost in resource-constrained global health settings.