Sparse Clustering with K-Means - Which Penalties and for Which Data?

While high dimensionality and the selection of meaningful features is usually a burden in machine learning, it is even more so in the case of unsupervised learning and particularly in clustering. The presence of uninformative features may bias significantly the results of distance-based methods such as K-means for instance. For tackling this issue, different versions of sparse K-means have been introduced, building on the idea of adding some penalty terms in the loss function and resulting into automatic feature selection and/or weighting. This paper investigates the connections between some of these methods, and particularly the differences induced by the choices of the penalty terms. It also illustrates the algorithms and model selection tools made available through a recently implemented R package, vimpclust.