Training with MAF, sampling with IAF

[EthanMarx]

Is it possible to get both the training speed benefits of using a Masked Autoregressive Flow, and then after training, the sampling speed of the Inverse Autoregressive Flow? Thank you!

[francois-rozet]

Hello @EthanMarx 👋

I don't think there is an easy way to find an efficient inverse for an autoregressive flow.

However, it should be possible to train one! You could take your trained MAF $z = f(x)$ and train an IAF $x = g(z)$ to solve $x = g(f(x))$ or $z = f(g(z))$. Both training objectives have advantages. The former requires access to data points $x$, but no back-propagation through $f$. The latter requires back-propagation through $f$, but only needs latent (Gaussian) samples $z$.

[EthanMarx]

Thanks @francois-rozet! And the training process should in principle be faster than training an IAF from scratch?

[francois-rozet]

Yes it should be faster as we only evaluate the "latent to data" direction of the IAF and the "data to latent" direction of the MAF, which are both very efficient.

If you happen to try, I would love to know the results!

[EthanMarx]

Okay thanks a ton for the tip!