The evaluation of universities has traditionally relied on metrics, such as citation counts, impact factors and participation in research projects. The results of such evaluations can significantly contribute to the academic assessment and strategic planning. Open science has introduced new dimensions of evaluation, including transparency, reproducibility, and accessibility. The paper explores the evolving landscape of academic evaluation, considering the impact of open science. We highlight the benefits and challenges of incorporating open science principles into academia, discuss existing frameworks and propose modifications to already existing evaluation methods in order to take into account and assess the adoption of open science in the academic setting. More specifically, we have adapted a dynamic multicriteria decision aiding method and the underlying ontology with the criteria identified by our research. As a direct consequence, the evaluation method reflects more accurately the current academic environment, promoting a more comprehensive and forward-looking assessment of research contributions.