Decision-making policies have been extensively studied, leading to continuous advances in the specialized literature. The use of Machine Learning (ML) for scheduling tasks emerged as a research opportunity resulting in policy refinement techniques and context-aware mechanisms. Specifically, the existing data from previous executions and workloads archive compose a rich knowledge data used for creating Reinforcement Learning (RL) models. However, a fact stands out from Data Center (DC) administrators’ decisions; although research shows efficient policies, administrators continue to use simple scheduling policies based on variants of First Come First Served (FCFS), mainly motivated by a lack of explainability and confidence in long-term performance indicators of RL-based policies. In this context, this work sheds light on performance indicators of RL-based policies. The comparison is carried out in depth, seeking to demonstrate the real limitations of each one as well as the advantages. In addition to presenting a numerical analysis, we discuss the qualitative aspects of the approaches. The lessons learned are essential to assist DC administrators.