Group sparse sufficient dimension reduction: a model-free group variable selection method

In many scientific applications, the covariates fall naturally into different groups, for example, the genes can be grouped by biological pathways in biological studies. In this study, we propose a new model-free group variable selection method by introducing a novel penalty, called adaptive group composite penalty. The proposed method can simultaneously achieve both sufficient dimension reduction and group variable selection in the case of diverging number of covariates. It can also simultaneously select important individual and group variables in a model-free fashion. An iterative two-stage algorithm is built to carry out the proposed method by reformulating the penalized objective functions. We provide the penalized sufficient dimension reduction estimators that estimate the targeted central subspace, and study their asymptotic properties. Simulation studies show that the proposed method gains significant efficiency in dimension reduction and variable selection, and it outperforms the other classical sparse sufficient dimension reduction methods in removing unimportant covariates, especially the unimportant groups. We illustrate the proposed method using a data set of RNA splicing signals.