Named Entities are fundamental to understand and process natural language, yet mention detection—identifying which text spans refer to entities—remains a primary performance bottleneck. Current approaches require supervised training for each schema, while generalist extractors like GLiNER need schema specification at inference time. Recent mechanistic interpretability research suggests that LLMs may naturally learn to encode entity structures during pretraining through mechanisms like binding IDs. This motivates our central hypothesis: if LLMs naturally develop entity detection capabilities, we should be able to extract them efficiently without task-specific training or schema specification. This PhD research investigates four main questions: how LLMs represent multi-token entities, what mechanisms enable entity manipulation and relational reasoning, whether we can efficiently extract mentions from their hidden representations and how can these results finally enhance information retrieval. We propose using task vectors for entity mention reconstruction, and lightweight detection architectures for generalist mention detection. This work aims to enable interpretable entity processing, efficient extraction with lightweight models, and zero-shot transfer across datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Understanding LLM Entity Representations for Efficient and Interpretable Entity Detection

  • Victor Morand

摘要

Named Entities are fundamental to understand and process natural language, yet mention detection—identifying which text spans refer to entities—remains a primary performance bottleneck. Current approaches require supervised training for each schema, while generalist extractors like GLiNER need schema specification at inference time. Recent mechanistic interpretability research suggests that LLMs may naturally learn to encode entity structures during pretraining through mechanisms like binding IDs. This motivates our central hypothesis: if LLMs naturally develop entity detection capabilities, we should be able to extract them efficiently without task-specific training or schema specification. This PhD research investigates four main questions: how LLMs represent multi-token entities, what mechanisms enable entity manipulation and relational reasoning, whether we can efficiently extract mentions from their hidden representations and how can these results finally enhance information retrieval. We propose using task vectors for entity mention reconstruction, and lightweight detection architectures for generalist mention detection. This work aims to enable interpretable entity processing, efficient extraction with lightweight models, and zero-shot transfer across datasets.