Predicting extreme weather events requires a careful mix of inductive and deductive reasoning as well as an understanding of epistemic and aleatory uncertainties to inform effective decision-making in high-risk situations. Tropical cyclones are among the most devastating extreme events and can cause extensive environmental damage, especially when they reach coastlines and inhabited areas due to their wide geographic extent and destructive power. This paper provides insight into two white-box Machine Learning methods applied to tropical cyclone forecasting, focusing on the interpretability of their predictions and their ability to handle uncertainty: both aspects are fundamental for supporting any successive deductive inference and decision-making activity. We consider Decision Trees and Bayesian Rule Lists, which demonstrate effectiveness in describing the presence of a cyclone and achieving mutually consistent results, and we critically assess their strengths and weaknesses.