The increase in the number of conversational systems (chatbots) applied to different scenarios in society is notable. However, the development of metrics and evaluation methods for chatbots still remains an open line of research. The aim is to create evaluation methods that are less and less invasive and that do not reduce the participation of users or other human agents. In this work, in the methods section, a systematic review is carried out on different evaluation methods for chatbots. Then, in the same section, new metrics for the evaluation of conversations with chatbots inspired by the neutrosophic theory are presented. In the results section, the validation of the proposed model is carried out. The applicability of the model is evaluated and the proposal is subjected to expert triangulation methods. In the work, it is possible to demonstrate that the application of neutrosophic logic can contribute to achieving a more natural response from chatbots. This effect can be useful to mitigate the presence of false responses or hallucinations.