new issues in lecture notebooks

[bertdv]

The following issues have low priority but should be addressed for next year.

Prob theory

• Move https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Notational-Conventions to Gaussian lecture

• Rephrase key concept: "All valid probabilistic relations can be derived from just two fundamental principles: the sum rule and the product rule. These two rules form the foundation of probability theory, from which more complex constructs such as conditional probabilities, Bayes’ theorem, and marginalization naturally follow." (above https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Examples)

• add solution to exercise https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Moments-of-the-PDF

• We should not ask students to code in Julia before the Julia lecture. So, https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#Working-with-Distributions-in-code is inappropriate, as is the showing of code in coding examples before the Julia lecture.

• The example of the Gaussian product of Gaussian distributions is inappropriate here, since the Gaussian distribution has not yet been treated. Postpone to the Gaussian lecture. https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#PDF-for-the-Product-of-Two-Variables

• Inappropriate exercise formulation: "... from the elementary sum rule
  and the sum and product rules." at https://bmlip.github.io/course/lectures/Probability%20Theory%20Review.html#The-Sum-Rule-(**)

BML

• In BML class https://bmlip.github.io/course/lectures/Bayesian%20Machine%20Learning.html#2.-Parameter-Estimation, $x_n$ are not independent, but conditionally independent, given $\theta$.

• the data $D$ is hard-coded and does not adapt to the slider at the beginning of the lecture https://bmlip.github.io/course/lectures/Bayesian%20Machine%20Learning.html#Challenge-Revisited:-Predicting-a-Coin-Toss

• Let's not show too much Julia code yet before the Julia class.

• "Convergence to 0" is not an exercise. https://bmlip.github.io/course/lectures/Bayesian%20Machine%20Learning.html#Exercise:-Convergence-to-0

FFG

• In FFG class, add why large models are necessarily sparse!

Gaussians

• at the beginning of the class, add a link to the cheatsheet, which contains lots of Gaussian materials.

• add an animation for Bayes rule with Gaussian prior and Gaussian likelihood. The idea is to show that precisions add and precision-weighted means add. Let the user play with precisions in prior and likelihood.

• Update the estimation of $\theta$ from 1 measurement into a story of measuring your weight by # coconuts.

• Jaynes ref does not work consider http://www-biba.inrialpes.fr/Jaynes/prob.html instead

• Solution see later https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Solution must contain a forward link.

• remove refs to Roweis https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Linear-Transformations-of-Gaussian-Variables SRG-4a and SRG-8

• in proof of MLE for mean of Gaussian, https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Maximum-likelihood-estimate-of-mean, add a note to the cheatsheet in particular the Matrix algebra.

• The mini on Distributions in Julia is misplaced here https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Gaussian-Distributions-in-Julia . These are not gaussian distributions.

• add (3) model eval by evidence $p(D)$ and (4) application $p(x_\bullet|D)$ to learning about a Gaussian variable (after https://bmlip.github.io/course/lectures/The%20Gaussian%20Distribution.html#Inference)

Discrete data

• Move exercise on evidence for die toss to the main text.

Regression

• build model $p(y,x,w) = p(w) \prod_n p(y_n|x_n,w) \underbrace{p(x_n)}_{\delta(x_n - \hat{x}_n)}$

Classification

• Both generative and discriminative classification can be worked out from a model specification to lead to $$p(y_{\dot,k}=1 | x_\dot,D) \sim \mathrm{Cat}(\sigma(\beta_k^T x_\dot))$$

• query "Can you work out Bayesian logistic regression with Laplace approximation for the posterior on the weights" in chatGPT gives a nice calculation with shorter formulas (in matrix form) than currently in lecture notes.