As the usage of chatbots in business-to-customer contact grows, quality assessment becomes critical to avoiding post-deployment issues. The lack of standards and the difficulty of measuring natural language comprehension are problems for this examination. Furthermore, the existing research lacks objective and automatic strategies for improving chatbot design. To address this gap, this paper introduces the Chatbot Design Understanding (CDU), a framework for autonomous evaluation of chatbot design, generating inputs that highlight areas for development and validating the impact of changes. The CDU was evaluated in an experiment using real chatbots, proving its capacity to improve their performance, including significant increases in accuracy and F1-score and reductions in fallback and ambiguity rates, following the implementation.