Marginal Probability Maximization on Restricted Boltzmann Machines Based on Blocked Sampling
摘要
Restricted Boltzmann machines (RBMs) remain widely utilized as energy-based generative models owing to their simplicity, despite the growing prevalence of deep generative models. In various practical applications of RBMs, it is necessary to maximize the marginal distribution over the visible layer, which is generally intractable owing to its non-convexity and high dimensionality of the marginal distribution. When employing simulated annealing to address this problem, a crucial challenge is designing suitable transition probabilities at each inverse temperature. Existing approaches, such as Metropolis-Hastings, Gibbs sampling, and Langevin dynamics have significant limitations, including high computational costs and difficulties in tuning the hyperparameters. In this study, we propose a novel transition probability design based on blocked sampling between layers. Our method is derived by replicating hidden variables at integer inverse temperatures, inspired by the replica method in statistical mechanics. These replicas are then aggregated via the central limit theorem. The proposed transition scheme is fast, hyperparameter-free, and theoretically grounded. Experimental results on diverse real-world datasets demonstrate that the proposed method consistently achieves robust and high-quality optimization performance. This work provides a practical and principled solution to the underexplored problem of marginal probability maximization in RBMs.