Scholarlink

<p>Out-of-distribution data points diverge from the general profile of the data, typically defined by the specific task for which the machine learning model is being constructed. Machine learning models are more reliable when out-of-distribution detection is part of the pipeline. Out-of-domain detection models are employed not just to sieve input into a model, but also to scrutinise output from a generative model, a process known as selective generation [<CitationRef CitationID="CR1">1</CitationRef>]. In literature, Mahalanobis distance is widely used in anomaly detection. In this work, we leverage the relation of Mahalanobis distance to Hotelling’s T-squared and Chi-squared distribution, which is further adapted for the inference on out-of-domain detection task. Data is usually categorised into three types: a) in-domain, b) out-of-domain, and c) background data. We explore approaches: a) constructed solely with in-domain data, and b) constructed using both in-domain and background data. Our proposed approaches are background free and efficient, and shows promising results compared to existing work in the literature which employ background data. We show that Hotelling’s T-square approach improves upon the Chi-square approach.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Out-of-distribution detection in text using statistical techniques

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月26日

查看全文
  • Dhruv Raghavan,
  • Siddharth Bhandari,
  • Jyotirmoy Banerjee,
  • Subhadip Bandyopadhyay

摘要

Out-of-distribution data points diverge from the general profile of the data, typically defined by the specific task for which the machine learning model is being constructed. Machine learning models are more reliable when out-of-distribution detection is part of the pipeline. Out-of-domain detection models are employed not just to sieve input into a model, but also to scrutinise output from a generative model, a process known as selective generation [1]. In literature, Mahalanobis distance is widely used in anomaly detection. In this work, we leverage the relation of Mahalanobis distance to Hotelling’s T-squared and Chi-squared distribution, which is further adapted for the inference on out-of-domain detection task. Data is usually categorised into three types: a) in-domain, b) out-of-domain, and c) background data. We explore approaches: a) constructed solely with in-domain data, and b) constructed using both in-domain and background data. Our proposed approaches are background free and efficient, and shows promising results compared to existing work in the literature which employ background data. We show that Hotelling’s T-square approach improves upon the Chi-square approach.