<p>Coreset construction aims to identify a small, weighted subset of training data that approximates the full dataset’s statistical properties, thereby enabling efficient model training without significant performance loss. Existing coreset selection methods typically optimize for geometric coverage or mean matching, under the implicit assumption that these objectives transfer meaningfully to arbitrary downstream tasks. In this study, it is demonstrated that this assumption can fail when the downstream task depends on statistical moments that the selection objective often does not preserve. A common underlying phenomenon termed the <b>task-moment mismatch</b> is identified, wherein coreset objectives that neglect task-relevant statistical moments yield worse performance than uniform random sampling. The objective of this work is threefold: (1) to diagnose failure modes of coverage-based coreset selection on heavy-tailed distributions, (2) to characterize how optimizing for lower-order moments can degrade the preservation of higher-order statistical structure required by certain tasks, and (3) to provide a constructive demonstration that moment-aware selection can address this mismatch. A greedy selection procedure termed Hierarchical Moment-Preserving (HMP) selection is proposed, which targets specific moment tensors through a two-stage hierarchical approach. Experimental evaluation across synthetic distributions and realistic signals parameterized from real-world sources reveals that k-means++ exhibits <InlineEquation ID="IEq1"><EquationSource Format="TEX">\(5\times\)</EquationSource></InlineEquation> higher covariance error than random sampling on heavy-tailed data, confirming that coverage optimization can be counterproductive for moment-dependent tasks. Furthermore, covariance-optimized coresets are shown to cause <InlineEquation ID="IEq2"><EquationSource Format="TEX">\(-13.3\)</EquationSource></InlineEquation> dB signal separation degradation on Independent Component Analysis tasks (<InlineEquation ID="IEq3"><EquationSource Format="TEX">\(p = 0.002\)</EquationSource></InlineEquation>), attributable to the disruption of fourth-order cumulant structure that ICA requires. The proposed moment-aware approach achieves <InlineEquation ID="IEq4"><EquationSource Format="TEX">\(3\times\)</EquationSource></InlineEquation> lower covariance error than random sampling and <InlineEquation ID="IEq5"><EquationSource Format="TEX">\(+15\%\)</EquationSource></InlineEquation> improvement in outlier detection precision on financial data. A practical decision matrix is provided to guide practitioners in matching coreset methods to the statistical requirements of their downstream tasks.</p>

Task-moment mismatch: diagnosing failures of coverage-based methods on heavy-tailed distributions in coreset selection

期刊论文
出版模式：

开放获取
发表日期：

2026年8月5日

查看全文

Supreet Halagali,
Soumyalatha Naveen

摘要

Coreset construction aims to identify a small, weighted subset of training data that approximates the full dataset’s statistical properties, thereby enabling efficient model training without significant performance loss. Existing coreset selection methods typically optimize for geometric coverage or mean matching, under the implicit assumption that these objectives transfer meaningfully to arbitrary downstream tasks. In this study, it is demonstrated that this assumption can fail when the downstream task depends on statistical moments that the selection objective often does not preserve. A common underlying phenomenon termed the task-moment mismatch is identified, wherein coreset objectives that neglect task-relevant statistical moments yield worse performance than uniform random sampling. The objective of this work is threefold: (1) to diagnose failure modes of coverage-based coreset selection on heavy-tailed distributions, (2) to characterize how optimizing for lower-order moments can degrade the preservation of higher-order statistical structure required by certain tasks, and (3) to provide a constructive demonstration that moment-aware selection can address this mismatch. A greedy selection procedure termed Hierarchical Moment-Preserving (HMP) selection is proposed, which targets specific moment tensors through a two-stage hierarchical approach. Experimental evaluation across synthetic distributions and realistic signals parameterized from real-world sources reveals that k-means++ exhibits \(5\times\) higher covariance error than random sampling on heavy-tailed data, confirming that coverage optimization can be counterproductive for moment-dependent tasks. Furthermore, covariance-optimized coresets are shown to cause \(-13.3\) dB signal separation degradation on Independent Component Analysis tasks (\(p = 0.002\)), attributable to the disruption of fourth-order cumulant structure that ICA requires. The proposed moment-aware approach achieves \(3\times\) lower covariance error than random sampling and \(+15\%\) improvement in outlier detection precision on financial data. A practical decision matrix is provided to guide practitioners in matching coreset methods to the statistical requirements of their downstream tasks.