This paper describes the design and application of a new supplier evaluation and ranking tool that utilizes both multi-criteria decision-making methods and machine learning methods. The framework we propose addresses the important business problem of selecting suppliers, by utilizing user-defined criteria combined with AI-based evaluation methods. The framework allows for both types of supervised learning (if there are historical performance records for suppliers that could be labelled) and unsupervised clustering (if there are no historical performance records). Some other unique features offered by the framework are dynamic filtering, real-time details of comparisons between suppliers, and automated and comprehensive evaluation reports. The tool standardizes and encodes structural attributes of the supplier (e.g. cost, delivery time, technical specifications, etc.) no matter what types of data would be present (for example, nominal, ordinal, etc.). The experimental evaluation underscores that the system was flexible in adapting to diverse industry contexts, while also allowing several parameters to estimate identify the best suppliers (and the worst suppliers for that matter). In addition, the integration with NLP (Cohere API) provided leverage for automated knowledge synthesis about supplier strengths and weaknesses in addition to the numerical ranking provided in the profile created originally. This research contributes to the procurement management literature in a meaningful way as it provides a unique, scalable and practical solution that utilizes data science techniques to enhance a standard approach to supplier selection.