Learning Tree-Structured Distributions
摘要
Tree-structured probabilistic models offer a balance between interpretability and tractability, making them a cornerstone in density estimation and graphical model learning. In this work, we present a comprehensive empirical evaluation of three recent approaches for learning tree distributions, benchmarking them against the classic Chow–Liu algorithm. We provide the first practical implementations of two online learning methods—the Randomized Weighted Majority method and Online Forest Density Estimation—and introduce a novel variant that replaces matrix-tree sampling with Wilson’s loop-erased random walk algorithm for efficient spanning-tree generation. We further develop optimized implementations of the online learning methods to enable scalable evaluation on real-world datasets. On synthetic data, our results reveal a noise-dependent spectrum. Under moderate corruption, where the data remain close to a tree, Chow-Liu is hard to beat—attaining the best log-likelihood and the most accurate structure recovery; as the corruption grows, the online methods overtake it in log-likelihood, where their iterative updates and Bayesian smoothing pay off. On real-world benchmarks, the Wilson variant emerges as the strongest online method, matching Chow-Liu in aggregate log-likelihood while running orders of magnitude faster than the original online learner. Together, these results position the online methods—and RWM-Wilson in particular—as practical, scalable alternatives to the classical baseline when the tree assumption is violated, while confirming Chow–Liu’s enduring strength when it holds.