<p>Data-driven optimization utilizing machine learning has gained significant popularity in recent times. Nevertheless, machine learning methodologies often presuppose that the target variable of the dataset is uniformly distributed, leading to the imbalance problem. Classical approaches developed to address data imbalance are not suitable for application in the newsvendor problem due to the varying costs associated with over/under predictions. Additionally, there is a lack of appropriate metrics for selecting the correct model that accounts for imbalance in data-driven newsvendor problems. In this study, we propose a relevance-weighted (RW) learning framework adapted to deal with the imbalanced dataset and the newsvendor’s asymmetric costs, specifically by incorporating both demand rareness and over/under-prediction costs into a unified loss function. We also introduce the Newsvendor Error Cost Relevance Area (NECRA) metric, an adaptation of cumulative relevance-weighted metrics, specifically tailored for model selection under demand imbalance. Relevance-weighted learning allows researchers to construct a neural network model that assigns sample weights based on the rareness of demand values, thereby enabling the final model to predict rare demands more effectively than classical network models. We simulate an extensive amount of datasets with varying properties and compare our method to the classical data-driven newsvendor objective function. We analyze the findings using statistical tests and results confirm that relevance-weighted learning performs better for the imbalanced datasets.</p>