Given n noisy samples with p dimensions, where <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$n \ll p$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>n</mi> <mo>≪</mo> <mi>p</mi> </mrow> </math></EquationSource> </InlineEquation>, we show that the multi-step thresholding procedure based on the Lasso – we call it the Thresholded Lasso, can accurately estimate a sparse vector <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$\beta \in {\mathbb {R}}^p$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>β</mi> <mo>∈</mo> <msup> <mrow> <mi mathvariant="double-struck">R</mi> </mrow> <mi>p</mi> </msup> </mrow> </math></EquationSource> </InlineEquation> in a linear model <InlineEquation ID="IEq4"> <EquationSource Format="TEX">$Y = X \beta + \epsilon$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>Y</mi> <mo>=</mo> <mi>X</mi> <mi>β</mi> <mo>+</mo> <mi>ϵ</mi> </mrow> </math></EquationSource> </InlineEquation>, where X is a design matrix and <InlineEquation ID="IEq5"> <EquationSource Format="TEX">$\epsilon \sim N(0, \sigma ^2 I_n)$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>ϵ</mi> <mo>∼</mo> <mi>N</mi> <mo stretchy="false">(</mo> <mn>0</mn> <mo>,</mo> <msup> <mi>σ</mi> <mn>2</mn> </msup> <msub> <mi>I</mi> <mi>n</mi> </msub> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation>. Here <InlineEquation ID="IEq6"> <EquationSource Format="TEX">$I_n$</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>I</mi> <mi>n</mi> </msub> </math></EquationSource> </InlineEquation> denotes the identity matrix. We show that under the restricted eigenvalue condition, it is possible to achieve the <InlineEquation ID="IEq7"> <EquationSource Format="TEX">$\ell _2$</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>ℓ</mi> <mn>2</mn> </msub> </math></EquationSource> </InlineEquation> loss within a logarithmic factor of the ideal mean square error one would achieve with an oracle while selecting a sufficiently sparse model – hence achieving sparse oracle inequalities; the oracle would supply perfect information about which coordinates are non-zero and which are above the noise level. We also show the same property holds for the Gauss-Dantzig selector under a uniform uncertainty principle. Our simulation results match our theoretical analysis excellently.

Thresholded Lasso for high dimensional variable selection

期刊论文
发表日期：

2025年12月8日

查看全文

Shuheng Zhou

摘要

Given n noisy samples with p dimensions, where $n \ll p$ $n ≪ p$ , we show that the multi-step thresholding procedure based on the Lasso – we call it the Thresholded Lasso, can accurately estimate a sparse vector $\beta \in {\mathbb {R}}^p$ $β \in R^{p}$ in a linear model $Y = X \beta + \epsilon$ $Y = X β + ϵ$ , where X is a design matrix and $\epsilon \sim N(0, \sigma ^2 I_n)$ $ϵ \sim N (0, σ^{2} I_{n})$ . Here $I_n$ $I_{n}$ denotes the identity matrix. We show that under the restricted eigenvalue condition, it is possible to achieve the $\ell _2$ $ℓ_{2}$ loss within a logarithmic factor of the ideal mean square error one would achieve with an oracle while selecting a sufficiently sparse model – hence achieving sparse oracle inequalities; the oracle would supply perfect information about which coordinates are non-zero and which are above the noise level. We also show the same property holds for the Gauss-Dantzig selector under a uniform uncertainty principle. Our simulation results match our theoretical analysis excellently.