Update notebook

Author: ferrine

docs/source/notebooks/variational_api_quickstart.ipynb

@@ -37,7 +37,7 @@
    "metadata": {},
    "source": [
     "## Basic setup\n",
-    "We do not need comples models to play with VI API, instead we'll have a simple mixture model"
+    "We do not need complex models to play with VI API, instead we'll have a simple mixture model"
    ]
   },
   {
@@ -61,7 +61,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We can't compute analitical expectations quickly here. Instead we can get approximations for it with MC methods. Lets use NUTS first. It required these variables to be in deterministic list"
+    "We can't compute analytical expectations quickly here. Instead we can get approximations for it with MC methods. Lets use NUTS first. It required these variables to be in deterministic list"
    ]
   },
   {
@@ -124,9 +124,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Looks good, we can see multomodality matters. Moreover we have samples for $x^2$ and $sin(x)$. Mut running MCMC every time we need to calculate an expression is not the thing needed in many experiments. Moreover you should know in advance what exactly you want to see in trace and call `Deterministic(.)` on it. \n",
+    "Looks good, we can see multomodality matters. Moreover we have samples for $x^2$ and $sin(x)$. There is one drawback, you should know in advance what exactly you want to see in trace and call `Deterministic(.)` on it.\n",
     "\n",
-    "VI API is about the opposite approach. First You have a model, you don inference on it, then experiments comming. Let's do the same setup without deterministics"
+    "VI API is about the opposite approach. You do inference on model, then experiments come after. Let's do the same setup without deterministics"
    ]
   },
   {
@@ -405,6 +405,40 @@
     "advi.approx"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Different approximations have different parameters. In MeanField case we use $\\rho$ and $\\mu$ (inspired by [Bayes by BackProp](https://arxiv.org/abs/1505.05424))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 59,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'mu': mu, 'rho': rho}"
+      ]
+     },
+     "execution_count": 59,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "advi.approx.shared_params"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "But having convinient shortcuts happens to be usefull sometimes. One of most frequent cases is specifying mass matrix for NUTS"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 16,
@@ -446,6 +480,55 @@
     ")"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": 58,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "    Helper class to record arbitrary stats during VI\n",
+      "\n",
+      "    It is possible to pass a function that takes no arguments\n",
+      "    If call fails then (approx, hist, i) are passed\n",
+      "\n",
+      "\n",
+      "    Parameters\n",
+      "    ----------\n",
+      "    kwargs : key word arguments\n",
+      "        keys mapping statname to callable that records the stat\n",
+      "\n",
+      "    Examples\n",
+      "    --------\n",
+      "    Consider we want time on each iteration    \n",
+      "    >>> import time\n",
+      "    >>> tracker = Tracker(time=time.time)\n",
+      "    >>> with model:\n",
+      "    ...     approx = pm.fit(callbacks=[tracker])\n",
+      "    \n",
+      "    Time can be accessed via :code:`tracker['time']` now\n",
+      "    For more complex summary one can use callable that takes\n",
+      "    (approx, hist, i) as arguments\n",
+      "    >>> with model:\n",
+      "    ...     my_callable = lambda ap, h, i: h[-1]\n",
+      "    ...     tracker = Tracker(some_stat=my_callable)\n",
+      "    ...     approx = pm.fit(callbacks=[tracker])\n",
+      "    \n",
+      "    Multiple stats are valid too\n",
+      "    >>> with model:\n",
+      "    ...     tracker = Tracker(some_stat=my_callable, time=time.time)\n",
+      "    ...     approx = pm.fit(callbacks=[tracker])\n",
+      "    \n"
+     ]
+    }
+   ],
+   "source": [
+    "print(pm.callbacks.Tracker.__doc__)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -511,7 +594,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "That picture is very informative. We can see how poor mean converges and thar different value for it do not change elbo significantly. As we are using OO API, we can continue inference to get some visual convergence"
+    "That picture is very informative. We can see how poor mean converges and that different values for it do not change elbo significantly. As we are using OO API, we can continue inference to get some visual convergence"
    ]
   },
   {
@@ -848,7 +931,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Every time we get different value for the same theano node. That is because it is stochastic"
+    "Every time we get different value for the same theano node. That is because it is stochastic. After replacements we are free and do not depend on pymc3 model. We now depend on approximation. Changing it will change the distribution for stochastic nodes"
    ]
   },
   {
@@ -1006,12 +1089,57 @@
     "ass_.mean(0).eval()  # mean"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Symbolic sample size is OK too"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 61,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "i = theano.tensor.iscalar('i')\n",
+    "i.tag.test_value = 1\n",
+    "ass2_ = svgd_approx.sample_node(a, size=i)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 63,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "((100,), (10000,))"
+      ]
+     },
+     "execution_count": 63,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "ass2_.eval({i: 100}).shape, ass2_.eval({i: 10000}).shape"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "But unfortunately only scalar size is supported."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "### creating custom fuction\n",
-    "What about mode_replacements argument, it can be very usefull sometimes. Suppose you have th following setup:"
+    "What about mode_replacements argument, it can be very usefull sometimes. Suppose you have the following setup:"
    ]
   },
   {