Update variational inference interface to match [email protected]

[Red-Portal]

This PR does the following:

• The interface to AdvancedVI is updated to match the new v0.6 version.
• The way vi calls optimize is slightly changed so that algorithms that expect an unconstrained space can be used without being aware of Bijectors. In particular, the user is now expected to provide a q <: Bijectors.TransformedDistribution. vi will then unwrap the underlying unconstrained distribution, which is passed to AdvancedVI. Once the algorithm terminates, the result will be wrapped back and then returned.
• The deprecated v0.2-era interface is removed.

[github-actions]

Turing.jl documentation for PR #2699 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2699/

[codecov]

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.56%. Comparing base (0eb8576) to head (115802d).
⚠️ Report is 1 commits behind head on breaking.

Files with missing lines	Patch %	Lines
src/variational/VariationalInference.jl	0.00%	2 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (0eb8576) and HEAD (115802d). Click for more details.

HEAD has 6 uploads less than BASE

Flag	BASE (0eb8576)	HEAD (115802d)
	23	17

Additional details and impacted files

@@              Coverage Diff              @@
##           breaking    #2699       +/-   ##
=============================================
- Coverage     86.45%   58.56%   -27.90%     
=============================================
  Files            21       20        -1     
  Lines          1418     1395       -23     
=============================================
- Hits           1226      817      -409     
- Misses          192      578      +386     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mhauru]

Thanks @Red-Portal.

Since this changes the user-interface, could you please set the target branch of this PR to be breaking? We use that branch to collect breaking changes into a single, larger release. Currently breaking points at the same commit as main, so you shouldn't have to mess with git history for this.

In that same process, could you also please add an entry in HISTORY.md describing to users all interface changes (arguments, return values, behaviour) and what they can do to adapt to them. Maybe a code example of "this is how you used to call vi, this is how you do the same thing now", that sort of thing.

[penelopeysm]

I'm actually sorry to butt in on this, but are there new features available in 0.5, or is it just an interface change?

[Red-Portal]

@penelopeysm No new features.

[penelopeysm]

Okay. The reason why I asked is because this is an interface change for Turing and that is kind of annoying for end users if they don't get any benefit. I wonder in that case if it is worth doing either:

1. Not bumping the compat in Turing, waiting for the next release of AdvancedVI that has new features; or

2. Mapping the existing Turing interface to the new AdvancedVI interface, so that it's not a breaking change. I don't know how possible this is.

Other than that, I second everything that @mhauru brought up here, so I won't also review the details of this.

[Red-Portal]

@penelopeysm That's reasonable. I think we could go with the following for now:

Not bumping the compat in Turing, waiting for the next release of AdvancedVI that has new features;

[github-actions]

[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Turing.jl/docs/src/api.md

Lines 111 to 122 in eaac4c3

	| Exported symbol | Documentation | Description |
	|:---------------------- |:------------------------------------------------- |:---------------------------------------------------------------------------------------- |
	| `vi` | [`Turing.vi`](@ref) | Perform variational inference |
	| `q_locationscale` | [`Turing.Variational.q_locationscale`](@ref) | Find a numerically non-degenerate initialization for a location-scale variational family |
	| `q_meanfield_gaussian` | [`Turing.Variational.q_meanfield_gaussian`](@ref) | Find a numerically non-degenerate initialization for a mean-field Gaussian family |
	| `q_fullrank_gaussian` | [`Turing.Variational.q_fullrank_gaussian`](@ref) | Find a numerically non-degenerate initialization for a full-rank Gaussian family |
	| `KLMinRepGradDescent` | [`Turing.Variational.KLMinRepGradDescent`](@ref) | KL divergence minimization via stochastic gradient descent with the reparameterization gradient |
	| `KLMinRepGradProxDescent` | [`Turing.Variational.KLMinRepGradProxDescent`](@ref) | KL divergence minimization via stochastic proximal gradient descent with the reparameterization gradient over location-scale variational families |
	| `KLMinScoreGradDescent` | [`Turing.Variational.KLMinScoreGradDescent`](@ref) | KL divergence minimization via stochastic gradient descent with the score gradient |
	| `KLMinWassFwdBwd` | [`Turing.Variational.KLMinWassFwdBwd`](@ref) | KL divergence minimization via Wasserstein proximal gradient descent |
	| `KLMinNaturalGradDescent` | [`Turing.Variational.KLMinNaturalGradDescent`](@ref) | KL divergence minimization via natural gradient descent |
	| `KLMinSqrtNaturalGradDescent` | [`Turing.Variational.KLMinSqrtNaturalGradDescent`](@ref) | KL divergence minimization via natural gradient descent in the square-root parameterization |