The field of Artificial Intelligence (AI) has seen an exponential growth in recent years, with new models being created that achieve almost human-level accuracy in various tasks. However, this increase in accuracy sacrifices a key aspect for critical fields like medicine and finance, which is interpretability. This led to the rise of Explainability in Artificial Intelligence (XAI), which works towards giving more trustworthiness and transparency to these black-box models by generating explanations that translate the decision-making process of these models. One of the posing challenges in this field is to properly quantitatively evaluate these XAI methods due to limited research. This systematic review gives an overview of the current trends in the state-of-the-art for quantitative evaluation methods for evaluating XAI methods, giving more insight on what a good explanation should be and what properties it should meet. Then, we propose an evaluation benchmark based on the reviewed literature, comprising the properties we deemed most important to be met by an explanation. Finally, we present the limitations and challenges this field still poses and directions for future research.