<p>This paper concentrates on statistical models that feature both main effects and interaction terms, where the main effects exhibit a non-sparse structure while the interactions are sparse. This type of data structure is prevalent in various disciplines, including biology and genetics, and it often poses challenges due to the intricate interplay of interactions. We introduce a two-stage methodological method to address these modeling complexities, ensuring both accurate modeling and prediction. Our proposed method adeptly handles the complex correlations among interaction terms. Both numerical and theoretical evidence support the efficacy of our method across diverse problem domains. In extensive numerical comparisons with a range of established techniques, our method demonstrates superior statistical properties. We further apply our algorithm to a protein dataset related to Alzheimer’s disease, providing a practical tool for biological diagnosis.</p>