Training Algorithms for Mixtures of Normalizing Flows

In this paper, we focus on how a probabilistic mixture of normalizing flows can be fitted. In the literature, there are (at least) four approaches that do not necessarily provide an actual implementation of the method. These four algorithms are gradient ascent maximizing the log-likelihood of the data, (soft) expectation–maximization, hard expectation–maximization, and gradient ascent maximizing the evidence lower bound. Our contribution or the novelty of the paper can be described as follows: we (re)implement each method, we create a software program that encompasses all these four implementations, and we compare those on toy datasets and image datasets on which we fit a mixture of masked autoregressive flows. The non-linear flexibility is shown in the plots. The metrics and the running times are reported. There is not necessarily a certain training algorithm to be preferred, although there are some advantages and disadvantages for each. The code is available at https://github.com/aciobanusebi/training-algs-for-mnf .