Scholarlink

Recently, a wide range of services utilizing Large Language Models (LLMs) exist. However, many of these models offer similar functionality, making it difficult for users to identify the most suitable LLM for a given query. Moreover, the optimal choice of LLM often depends on both the specific task and the individual user preferences, further complicating the selection process even further. In this paper, we present a method for evaluating LLMs and dynamically routing user queries to the most appropriate model. The proposed system employs a Root LLM that generates diagnostic questions based on a user’s task description. A set of candidate LLMs then respond to these questions, and the Root LLM evaluates their answers to identify the model best suited for the task. Our evaluation shows that the root LLM correlates with human evaluations. Experimental results indicate that the Root LLM’s evaluations show a strong correlation with human judgments. Furthermore, the proposed system supports context-aware and user-adaptive routing, thereby improving both the relevance and quality of LLM-generated outputs.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

LLM-Based Evaluation for Dynamic Routing Among Large Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Takeshi Tsuchiya,
  • Ryuichi Mochizuki,
  • Hiroo Hirose,
  • Hiroshi Ichikawa,
  • Quang Tran Minh

摘要

Recently, a wide range of services utilizing Large Language Models (LLMs) exist. However, many of these models offer similar functionality, making it difficult for users to identify the most suitable LLM for a given query. Moreover, the optimal choice of LLM often depends on both the specific task and the individual user preferences, further complicating the selection process even further. In this paper, we present a method for evaluating LLMs and dynamically routing user queries to the most appropriate model. The proposed system employs a Root LLM that generates diagnostic questions based on a user’s task description. A set of candidate LLMs then respond to these questions, and the Root LLM evaluates their answers to identify the model best suited for the task. Our evaluation shows that the root LLM correlates with human evaluations. Experimental results indicate that the Root LLM’s evaluations show a strong correlation with human judgments. Furthermore, the proposed system supports context-aware and user-adaptive routing, thereby improving both the relevance and quality of LLM-generated outputs.