Demand forecasting is crucial in supply chain management for effectively matching supply with demand. Despite the common use of simple forecasting methods in practice, advanced statistical and machine learning techniques are known to offer improvements in accuracy. However, their impact on supply chain performance remains relatively underexplored. This paper compares the accuracy and performance of four forecasting approaches for six demand time series with trend and seasonality: moving average, seasonal naïve, triple exponential smoothing (TES), and Extreme Gradient Boosting (XGBoost). We demonstrate that the considerable accuracy improvements achieved by advanced methods (TES and XGBoost) lead to notable enhancements in supply chain performance metrics. Nevertheless, we also observe that as the differences in forecast errors decrease, improving accuracy does not always translate to higher performance. In our experiments, XGBoost minimises lost sales, while TES reduces inventory investments.