Generalized Dirichlet compound negative multinomial mixture models and applications

Burstiness, overdispersion, sparsity, and interfeature correlation in count vectors remain central challenges for mixture modeling. In this paper, the generalized Dirichlet distribution is proposed as a prior to the negative multinomial within a hierarchical Bayesian framework for clustering high-dimensional, overdispersed discrete data. The generalized Dirichlet prior introduces a richer covariance structure that captures both positive and negative feature correlations and enables feature-specific variance control, while preserving conjugacy and analytical tractability. To efficiently estimate mixture model parameters, we compare second-order Newton–Raphson and first-order gradient ascent updates with a minorization–maximization (MM) framework. MM optimizes a surrogate of the observed data log-likelihood, guarantees monotone ascent, requires no cumbersome calculations of gradients or Hessians, and preserves simplex and positivity constraints by construction. Empirically, MM reduces tuning burden and training time while reaching the same stationary solutions. Model selection is further addressed using the minimum message length (MML) criterion. Extensive experiments on real-world text, image, and video datasets show that combining the flexibility of the generalized Dirichlet prior with MM optimization and MML selection yields a mixture model that is statistically expressive, computationally stable, and practically effective.