Deploy preview for PR 612

Author: penelopeysm

pr-previews/612/search.json

@@ -316,7 +316,7 @@
     "href": "tutorials/variational-inference/index.html#basic-usage",
     "title": "Variational Inference",
     "section": "Basic Usage",
-    "text": "Basic Usage\nTo run VI, we must first set a variational family. For instance, the most commonly used family is the mean-field Gaussian family. For this, Turing provides functions that automatically construct the initialization corresponding to the model m:\n\nq_init = q_meanfield_gaussian(m);\n\nvi will automatically recognize the variational family through the type of q_init. Here is a detailed documentation for the constructor:\n\n@doc(Variational.q_meanfield_gaussian)\n\nq_meanfield_gaussian(\n    [rng::Random.AbstractRNG,]\n    model::DynamicPPL.Model;\n    location::Union{Nothing,<:AbstractVector} = nothing,\n    scale::Union{Nothing,<:Diagonal} = nothing,\n    kwargs...\n)\nFind a numerically non-degenerate mean-field Gaussian q for approximating the  target model.\nArguments\n\nmodel: The target DynamicPPL.Model.\n\n\nKeyword Arguments\n\nlocation: The location parameter of the initialization. If nothing, a vector of zeros is used.\n\nscale: The scale parameter of the initialization. If nothing, an identity matrix is used.\n\n\nThe remaining keyword arguments are passed to q_locationscale.\nReturns\n\nq::Bijectors.TransformedDistribution: A AdvancedVI.LocationScale distribution matching the support of model.\n\n\n\n\n\n\n\nAs we can see, the precise initialization can be customized through the keyword arguments.\nLet’s run VI with the default setting:\n\nn_iters = 1000\nq_avg, q_last, info, state = vi(m, q_init, n_iters; show_progress=false);\n\nThe default setting uses the AdvancedVI.RepGardELBO objective, which corresponds to a variant of what is known as automatic differentiation VI7 or stochastic gradient VI8 or black-box VI9 with the reparameterization gradient101112. The default optimizer we use is AdvancedVI.DoWG13 combined with a proximal operator. (The use of proximal operators with VI on a location-scale family is discussed in detail by J. Domke1415 and others16.) We will take a deeper look into the returned values and the keyword arguments in the following subsections. First, here is the full documentation for vi:\n\n@doc(Variational.vi)\n\nvi(\n    [rng::Random.AbstractRNG,]\n    model::DynamicPPL.Model;\n    q,\n    n_iterations::Int;\n    objective::AdvancedVI.AbstractVariationalObjective = AdvancedVI.RepGradELBO(\n        10; entropy = AdvancedVI.ClosedFormEntropyZeroGradient()\n    ),\n    show_progress::Bool = Turing.PROGRESS[],\n    optimizer::Optimisers.AbstractRule = AdvancedVI.DoWG(),\n    averager::AdvancedVI.AbstractAverager = AdvancedVI.PolynomialAveraging(),\n    operator::AdvancedVI.AbstractOperator = AdvancedVI.ProximalLocationScaleEntropy(),\n    adtype::ADTypes.AbstractADType = Turing.DEFAULT_ADTYPE,\n    kwargs...\n)\nApproximating the target model via variational inference by optimizing objective with the initialization q. This is a thin wrapper around AdvancedVI.optimize.\nArguments\n\nmodel: The target DynamicPPL.Model.\n\nq: The initial variational approximation.\n\nn_iterations: Number of optimization steps.\n\n\nKeyword Arguments\n\nobjective: Variational objective to be optimized.\n\nshow_progress: Whether to show the progress bar.\n\noptimizer: Optimization algorithm.\n\naverager: Parameter averaging strategy.\n\noperator: Operator applied after each optimization step.\n\nadtype: Automatic differentiation backend.\n\n\nSee the docs of AdvancedVI.optimize for additional keyword arguments.\nReturns\n\nq: Variational distribution formed by the last iterate of the optimization run.\n\nq_avg: Variational distribution formed by the averaged iterates according to averager.\n\nstate: Collection of states used for optimization. This can be used to resume from a past call to vi.\n\ninfo: Information generated during the optimization run.",
+    "text": "Basic Usage\nTo run VI, we must first set a variational family. For instance, the most commonly used family is the mean-field Gaussian family. For this, Turing provides functions that automatically construct the initialization corresponding to the model m:\n\nq_init = q_meanfield_gaussian(m);\n\nvi will automatically recognize the variational family through the type of q_init. Here is a detailed documentation for the constructor:\n\n@doc(Variational.q_meanfield_gaussian)\n\nq_meanfield_gaussian(\n    [rng::Random.AbstractRNG,]\n    model::DynamicPPL.Model;\n    location::Union{Nothing,<:AbstractVector} = nothing,\n    scale::Union{Nothing,<:Diagonal} = nothing,\n    kwargs...\n)\nFind a numerically non-degenerate mean-field Gaussian q for approximating the  target model.\nArguments\n\nmodel: The target DynamicPPL.Model.\n\n\nKeyword Arguments\n\nlocation: The location parameter of the initialization. If nothing, a vector of zeros is used.\n\nscale: The scale parameter of the initialization. If nothing, an identity matrix is used.\n\n\nThe remaining keyword arguments are passed to q_locationscale.\nReturns\n\nq::Bijectors.TransformedDistribution: A AdvancedVI.LocationScale distribution matching the support of model.\n\n\n\n\n\n\n\nAs we can see, the precise initialization can be customized through the keyword arguments.\nLet’s run VI with the default setting:\n\nn_iters = 1000\nq_avg, q_last, info, state = vi(m, q_init, n_iters; show_progress=false);\n\nThe default setting uses the AdvancedVI.RepGradELBO objective, which corresponds to a variant of what is known as automatic differentiation VI7 or stochastic gradient VI8 or black-box VI9 with the reparameterization gradient101112. The default optimizer we use is AdvancedVI.DoWG13 combined with a proximal operator. (The use of proximal operators with VI on a location-scale family is discussed in detail by J. Domke1415 and others16.) We will take a deeper look into the returned values and the keyword arguments in the following subsections. First, here is the full documentation for vi:\n\n@doc(Variational.vi)\n\nvi(\n    [rng::Random.AbstractRNG,]\n    model::DynamicPPL.Model;\n    q,\n    n_iterations::Int;\n    objective::AdvancedVI.AbstractVariationalObjective = AdvancedVI.RepGradELBO(\n        10; entropy = AdvancedVI.ClosedFormEntropyZeroGradient()\n    ),\n    show_progress::Bool = Turing.PROGRESS[],\n    optimizer::Optimisers.AbstractRule = AdvancedVI.DoWG(),\n    averager::AdvancedVI.AbstractAverager = AdvancedVI.PolynomialAveraging(),\n    operator::AdvancedVI.AbstractOperator = AdvancedVI.ProximalLocationScaleEntropy(),\n    adtype::ADTypes.AbstractADType = Turing.DEFAULT_ADTYPE,\n    kwargs...\n)\nApproximating the target model via variational inference by optimizing objective with the initialization q. This is a thin wrapper around AdvancedVI.optimize.\nArguments\n\nmodel: The target DynamicPPL.Model.\n\nq: The initial variational approximation.\n\nn_iterations: Number of optimization steps.\n\n\nKeyword Arguments\n\nobjective: Variational objective to be optimized.\n\nshow_progress: Whether to show the progress bar.\n\noptimizer: Optimization algorithm.\n\naverager: Parameter averaging strategy.\n\noperator: Operator applied after each optimization step.\n\nadtype: Automatic differentiation backend.\n\n\nSee the docs of AdvancedVI.optimize for additional keyword arguments.\nReturns\n\nq: Variational distribution formed by the last iterate of the optimization run.\n\nq_avg: Variational distribution formed by the averaged iterates according to averager.\n\nstate: Collection of states used for optimization. This can be used to resume from a past call to vi.\n\ninfo: Information generated during the optimization run.",
     "crumbs": [
       "Get Started",
       "Tutorials",
@@ -352,7 +352,7 @@
     "href": "tutorials/variational-inference/index.html#using-different-optimisers",
     "title": "Variational Inference",
     "section": "Using Different Optimisers",
-    "text": "Using Different Optimisers\nThe default optimiser we use is a proximal variant of DoWG18. For Gaussian variational families, this works well as a default option. Sometimes, the step size of AdvancedVI.DoWG could be too large, resulting in unstable behavior. (In this case, we recommend trying AdvancedVI.DoG19) Or, for whatever reason, it might be desirable to use a different optimiser. Our implementation supports any optimiser that implements the Optimisers.jl interface.\nFor instance, let’s try using Optimiers.Adam20, which is a popular choice. Since AdvancedVI does not implement a proximal operator for Optimisers.Adam, we must use the AdvancedVI.ClipScale() projection operator, which ensures that the scale matrix of the variational approximation is positive definite. (See the paper by J. Domke 202021 for more detail about the use of a projection operator.)\n\nusing Optimisers\n\n_, _, info_adam, _ = vi(m, q_init, n_iters; show_progress=false, callback=callback, optimizer=Optimisers.Adam(3e-3), operator=ClipScale());\n\n\niters     = 1:10:length(info_mf)\nelbo_adam = [i.elbo_avg for i in info_adam[iters]]\nPlots.plot(iters, elbo_mf, xlabel=\"Iterations\", ylabel=\"ELBO\", label=\"DoWG\")\nPlots.plot!(iters, elbo_adam, xlabel=\"Iterations\", ylabel=\"ELBO\", label=\"Adam\")\n\n\n\n\n  \n    \n  \n\n\n\n  \n    \n  \n\n\n\n  \n    \n  \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCompared to the default option AdvancedVI.DoWG(), we can see that Optimisers.Adam(3e-3) is converging more slowly. With more step size tuning, it is possible that Optimisers.Adam could perform better or equal. That is, most common optimisers require some degree of tuning to perform better or comparably to AdvancedVI.DoWG() or AdvancedVI.DoG(), which do not require much tuning at all. Due to this fact, they are referred to as parameter-free optimizers.",
+    "text": "Using Different Optimisers\nThe default optimiser we use is a proximal variant of DoWG18. For Gaussian variational families, this works well as a default option. Sometimes, the step size of AdvancedVI.DoWG could be too large, resulting in unstable behavior. (In this case, we recommend trying AdvancedVI.DoG19) Or, for whatever reason, it might be desirable to use a different optimiser. Our implementation supports any optimiser that implements the Optimisers.jl interface.\nFor instance, let’s try using Optimisers.Adam20, which is a popular choice. Since AdvancedVI does not implement a proximal operator for Optimisers.Adam, we must use the AdvancedVI.ClipScale() projection operator, which ensures that the scale matrix of the variational approximation is positive definite. (See the paper by J. Domke 202021 for more detail about the use of a projection operator.)\n\nusing Optimisers\n\n_, _, info_adam, _ = vi(m, q_init, n_iters; show_progress=false, callback=callback, optimizer=Optimisers.Adam(3e-3), operator=ClipScale());\n\n\niters     = 1:10:length(info_mf)\nelbo_adam = [i.elbo_avg for i in info_adam[iters]]\nPlots.plot(iters, elbo_mf, xlabel=\"Iterations\", ylabel=\"ELBO\", label=\"DoWG\")\nPlots.plot!(iters, elbo_adam, xlabel=\"Iterations\", ylabel=\"ELBO\", label=\"Adam\")\n\n\n\n\n  \n    \n  \n\n\n\n  \n    \n  \n\n\n\n  \n    \n  \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCompared to the default option AdvancedVI.DoWG(), we can see that Optimisers.Adam(3e-3) is converging more slowly. With more step size tuning, it is possible that Optimisers.Adam could perform better or equal. That is, most common optimisers require some degree of tuning to perform better or comparably to AdvancedVI.DoWG() or AdvancedVI.DoG(), which do not require much tuning at all. Due to this fact, they are referred to as parameter-free optimizers.",
     "crumbs": [
       "Get Started",
       "Tutorials",

pr-previews/612/sitemap.xml

@@ -2,166 +2,166 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
     <loc>https://turinglang.org/docs/tutorials/gaussian-process-latent-variable-models/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/probabilistic-pca/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/bayesian-poisson-regression/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/bayesian-neural-networks/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/gaussian-mixture-models/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/bayesian-time-series-analysis/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/bayesian-differential-equations/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.667Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/variational-inference/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/compiler/minituring-contexts/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.663Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/compiler/design-overview/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.536Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.663Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/inference/implementing-samplers/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/inference/abstractmcmc-interface/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/contributing/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/transforms/bijectors/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.538Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/getting-started/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.540Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.667Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/dynamichmc/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/sampler-visualisation/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.670Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/modifying-logprob/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/automatic-differentiation/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/mode-estimation/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/troubleshooting/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.670Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/external-samplers/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/probability-interface/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.670Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/custom-distribution/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/tracking-extra-quantities/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.670Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/usage/performance-tips/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.543Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.670Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/core-functionality/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.536Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.663Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/transforms/distributions/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.538Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.665Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/transforms/dynamicppl/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.540Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.667Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/inference/variational-inference/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.538Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/inference/abstractmcmc-turing/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/compiler/model-manual/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.663Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/compiler/minituring-compiler/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.663Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/developers/contexts/submodel-condition/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.537Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.664Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/bayesian-logistic-regression/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/bayesian-linear-regression/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.667Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/coin-flipping/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.541Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/hidden-markov-models/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/multinomial-logistic-regression/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/gaussian-processes-introduction/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.668Z</lastmod>
   </url>
   <url>
     <loc>https://turinglang.org/docs/tutorials/infinite-mixture-models/index.html</loc>
-    <lastmod>2025-06-23T16:17:22.542Z</lastmod>
+    <lastmod>2025-06-23T16:24:28.669Z</lastmod>
   </url>
 </urlset>