Analysis of Lack of Agreement Between MCDM Methods Related to the Solution of a Problem: Proposing a Methodology for Comparing Methods to a Reference

It is a proven fact that at present, there is no course of action that can evaluate or validate the reliability of the solution reached by a MCDM method, because the “true” solution is not known, and it is impossible to make a comparison to assess the efficiency of a result found. This Chapter presents a procedure that can help in this endeavor. It proposes to use a proxy of the true solution, to test a result of any MCDM method; this proxy solution must be the consequence of a more faithful model to replicate as much as possible real-world conditions, as well as the absence of subjectivity in criteria weighting, and the result achieved by an indisputable mathematical procedure. For this purpose, this book suggests using the SIMUS method that fulfils these conditions. In so doing, a problem is solved by this method and its result is used as a benchmark to determine the closeness to this result by other methods. To measure closeness to the proxy, it is suggested to use the Kendall Tau Rank Correlation Coefficient (Kendall, Biometrika 30(1–2):81–89, 1938).