Choosing Among PCA, FA, LCA, LPA, and LDA

Principal component analysis (PCA), factor analysis (FA), latent class analysis (LCA), and latent profile analysis (LPA) are prominent statistical models in the social and behavioral sciences, employed to reveal underlying patterns and relationships within extensive datasets. PCA involves reducing observed variables while retaining dataset variance by generating principal components. FA aims to decrease observed variable dimensions by identifying underlying factors. LCA and LPA, respectively, focus on discerning latent classes and profiles based on response patterns to categorical and continuous variables. Recently, latent Dirichlet allocation (LDA) method has gained increasing popularity for its application in analyzing textual data alongside numerical responses, contributing to the extraction of hidden structures within complex textual data. Topic modeling (TM), a broader method, is used to identify underlying themes or topics in text corpora, with LDA representing a specific application of TM. Despite some common goals of uncovering hidden structures, these models vary in statistical theory, algorithms, and applications. This paper seeks to investigate these differences and similarities, facilitating researchers in selecting the appropriate model to address their specific research questions. Each model is introduced first, outlining its statistical form and relevant theory. A comprehensive comparison of these models emphasizing their distinguishing features is provided, followed by the illustration of these models and methods. Finally, practical recommendations are offered to aid researchers in effectively implementing these models in their own research.