Pursuing the Perfect Projection: A Projection Pursuit Framework for Deep Learning

The curse of dimensionality refers to phenomena occurring with increasing dimensionality such as marginal differences in distances. Projection pursuit solves this issue by projecting high-dimensional data into a low-dimensional space where meaningful distances allow unbiased function estimation. However, projection pursuit only considers projections onto lines and the unbiased function depends on the sample size. We introduce deep projection pursuit (DPP) to remedy these limitations by using an ensemble of projections on parameterized surfaces combined with neural networks to solve the learning task. Furthermore, we demonstrate the capabilities of the DPP framework by training principal component curves and solving supervised tasks with interpretable models. Finally, we show the ability to maintain group properties in the projection space. Due to these applications, deep projection pursuit is a flexible design paradigm with various use cases.