<p>The determination of the most important states in Markov Decision Processes has recently gained interest due to its applicability in increasing both the performance and the explainability of Reinforcement Learning algorithms. In this paper, we extend the concept of Reconstructed Transition Graphs from simple unweighted graphs to advanced notions of weighted Multigraphs and Multiplexes to better represent Markov Decision Processes and identify the most important states. To this end, we adapt the classical centrality measures (betweenness, closeness, and eigenvector) to our proposed frameworks and delineate the theoretical assumptions required to ensure their existence and uniqueness. Compared to other graph-based measures available in the literature, this novel approach can take into account the different rewards and transform them into weights specific to the task. Computational experiments indicate the superior capacity of these measures in the detection of dangerous states and high-reward situations for non-sparse reward environments compared to state-of-the-art methodologies. Moreover, a statistical analysis carried out allowed to discover significant interconnections between the newly defined centrality measure structures and Q-value-based measures that are helpful for the recognition of critical states.</p>