Expertise matching plays a pivotal role in academic peer review and industrial innovation. However, existing research suffers from three critical limitations: (1) a predominant focus on matching papers to reviewers, overlooking the equally important reverse direction—matching reviewers to papers—which is essential for reviewer engagement; (2) confinement to academic domains, which hinders cross-domain generalization; and (3) a fragmented evaluation landscape that precludes fair and systematic model comparison. To address these gaps, we present ExperMatch, the first unified benchmark for expertise matching that supports bidirectional matching (e.g., papers \(\leftrightarrow \) reviewers) as well as across domains (e.g., academia vs. industry). Built on rigorously curated datasets with graded relevance labels, ExperMatch enables comprehensive evaluation across diverse expertise matching paradigms. Our extensive experiments reveal a fundamental trade-off between matching effectiveness and computational efficiency: while dense embedding models achieve the highest accuracy, they incur prohibitive inference costs. In contrast, a context-aware multi-factor model delivers competitive performance at substantially lower latency. These results demonstrate that the optimal model choice is highly dependent on task-specific requirements. Notably, ExperMatch ’s insights have been deployed on a large-scale innovation platform at a leading tech company, demonstrating real-world impact.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ExperMatch: A Unified Benchmark for Bidirectional and Cross-Domain Expertise Matching

  • Wei Chen,
  • Kaibin Chen,
  • Yu-Xuan Qiu,
  • Weipeng Zhang,
  • Minhua Lu,
  • Qianting Chen,
  • Jiuzhang Liu,
  • Wai Kin Chan,
  • Rui Mao

摘要

Expertise matching plays a pivotal role in academic peer review and industrial innovation. However, existing research suffers from three critical limitations: (1) a predominant focus on matching papers to reviewers, overlooking the equally important reverse direction—matching reviewers to papers—which is essential for reviewer engagement; (2) confinement to academic domains, which hinders cross-domain generalization; and (3) a fragmented evaluation landscape that precludes fair and systematic model comparison. To address these gaps, we present ExperMatch, the first unified benchmark for expertise matching that supports bidirectional matching (e.g., papers \(\leftrightarrow \) reviewers) as well as across domains (e.g., academia vs. industry). Built on rigorously curated datasets with graded relevance labels, ExperMatch enables comprehensive evaluation across diverse expertise matching paradigms. Our extensive experiments reveal a fundamental trade-off between matching effectiveness and computational efficiency: while dense embedding models achieve the highest accuracy, they incur prohibitive inference costs. In contrast, a context-aware multi-factor model delivers competitive performance at substantially lower latency. These results demonstrate that the optimal model choice is highly dependent on task-specific requirements. Notably, ExperMatch ’s insights have been deployed on a large-scale innovation platform at a leading tech company, demonstrating real-world impact.