Agentic AI systems can access vast data but struggle to apply domain expertise, namely the contextual understanding of how to use specialized information. This paper presents a practical framework for encoding such expertise, demonstrated with the National Football League (NFL) through NFL Fantasy AI, a production system delivering analyst-grade fantasy football advice, as assessed by NFL Pro analysts. We introduce a three-step encoding method: (1) analyst-sourced reasoning guidance, encoding analytical patterns as generalized guidance rather than enumerated rules; (2) category-level semantic framing, where experts describe data usage rather than definitions; and (3) LLM-optimized semantic interfaces, using model-to-model iteration for field naming and tool design. Deployed in eight weeks, the system achieved over 90% analyst agreement on response quality, sub-5-s response times, and zero policy violations across more than 10,000 production queries.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Encoding Domain Expertise in Agents: Lessons from NFL Fantasy AI

  • Michael Butler,
  • Henry Wang,
  • Jake Lee,
  • Kenton Blacutt,
  • Dan Volk,
  • Mike Band,
  • Diego Socolinsky

摘要

Agentic AI systems can access vast data but struggle to apply domain expertise, namely the contextual understanding of how to use specialized information. This paper presents a practical framework for encoding such expertise, demonstrated with the National Football League (NFL) through NFL Fantasy AI, a production system delivering analyst-grade fantasy football advice, as assessed by NFL Pro analysts. We introduce a three-step encoding method: (1) analyst-sourced reasoning guidance, encoding analytical patterns as generalized guidance rather than enumerated rules; (2) category-level semantic framing, where experts describe data usage rather than definitions; and (3) LLM-optimized semantic interfaces, using model-to-model iteration for field naming and tool design. Deployed in eight weeks, the system achieved over 90% analyst agreement on response quality, sub-5-s response times, and zero policy violations across more than 10,000 production queries.