Exploring Non-convex Optimization in Sparse Signal Recovery: A Comparative Study of Non-convex Dantzig Selector and LASSO

Sparse signal recovery, lying at the core of signal processing and machine learning, has been traditionally attacked with convex optimization approaches such as LASSO and the Dantzig Selector. This paper investigates the field of non-convex optimization for sparse signal recovery, particularly comparing non-convex variants of Dantzig Selector and LASSO. A motivating choice of non-convex penalties may thus capture more complicated structures in sparse signals, offering a flexible framework for modeling complex sparsity patterns. Evaluate the performance of a non-convex Dantzig Selector in comparison with a non-convex variant of LASSO and investigate the sensitivity of the parameters in both methods. The non-convex Dantzig Selector introduces Huber loss, whereas the non-convex LASSO introduces a square root penalty. The performance metrics include sparsity and accuracy of recovered signals. The comparison between those methods in different scenarios gives the relative merits, adding up to the discussion on what methods to use for the reconstruction of signals. The study finds that the non-convex Dantzig Selector is of the final sparsity of the order of 10%, basically selecting only 10% of the variables. Contrarily, the non-convex LASSO is of about 90% sparsity, selecting about 90% of the variables. Finally, the non-convex LASSO has final accuracy closer to 1 with low prediction error, while the non-convex Dantzig Selector has final accuracy closer to 0.6, pointing to the reality of higher prediction error.