Spare parts demand forecasting is a central element of smart maintenance. However, it remains a challenging and often neglected task. Complex demand patterns complicate the use of conventional forecasting methods and require more advanced approaches. This study investigates whether data-driven methods based on machine learning and deep learning can improve forecasting performance compared to simple baseline models. A central objective is to examine how the choice of loss function during model training influences forecasting results when evaluated with different error measures. Using a real-world dataset of spare parts demand time series, we provide a systematic benchmark analysis. The findings advance existing research by highlighting the importance of aligning training objectives with evaluation measures, particularly for the use case of spare part demand, and contribute to the broader field of time series forecasting.