You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: lectures/ar1_bayes.md
+5-15Lines changed: 5 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,15 +37,12 @@ import matplotlib.pyplot as plt
37
37
38
38
This lecture uses Bayesian methods offered by [`numpyro`](https://num.pyro.ai/en/stable/) to make statistical inferences about two parameters of a univariate first-order autoregression.
39
39
40
-
41
40
The model is a good laboratory for illustrating the
42
41
consequences of alternative ways of modeling the distribution of the initial $y_0$:
43
42
44
43
- As a fixed number
45
-
46
44
- As a random variable drawn from the stationary distribution of the $\{y_t\}$ stochastic process
47
45
48
-
49
46
The first component of the statistical model is
50
47
51
48
$$
@@ -62,8 +59,6 @@ $$
62
59
y_0 \sim {\mathcal{N}}(\mu_0, \sigma_0^2)
63
60
$$ (eq:themodel_2)
64
61
65
-
66
-
67
62
Consider a sample $\{y_t\}_{t=0}^T$ governed by this statistical model.
68
63
69
64
The model implies that the likelihood function of $\{y_t\}_{t=0}^T$ can be *factored*:
@@ -88,7 +83,6 @@ We want to study how inferences about the unknown parameters $(\rho, \sigma_x)$
88
83
Below, we study two widely used alternative assumptions:
89
84
90
85
- $(\mu_0,\sigma_0) = (y_0, 0)$ which means that $y_0$ is drawn from the distribution ${\mathcal N}(y_0, 0)$; in effect, we are *conditioning on an observed initial value*.
91
-
92
86
- $\mu_0,\sigma_0$ are functions of $\rho, \sigma_x$ because $y_0$ is drawn from the stationary distribution implied by $\rho, \sigma_x$.
93
87
94
88
@@ -125,17 +119,13 @@ Basically, when $y_0$ happens to be in the tail of the stationary distribution a
125
119
126
120
An example below shows how not conditioning on $y_0$ adversely shifts the posterior probability distribution of $\rho$ toward larger values.
127
121
128
-
129
122
We begin by solving a *direct problem* that simulates an AR(1) process.
130
123
131
124
How we select the initial value $y_0$ matters:
132
125
133
-
* If we think $y_0$ is drawn from the stationary distribution ${\mathcal N}(0, \frac{\sigma_x^{2}}{1-\rho^2})$, then it is a good idea to use this distribution as $f(y_0)$.
134
-
135
-
- Why? Because $y_0$ contains information about $\rho, \sigma_x$.
136
-
137
-
* If we suspect that $y_0$ is far in the tail of the stationary distribution -- so that variation in early observations in the sample has a significant *transient component* -- it is better to condition on $y_0$ by setting $f(y_0) = 1$.
138
-
126
+
* If we think $y_0$ is drawn from the stationary distribution ${\mathcal N}(0, \frac{\sigma_x^{2}}{1-\rho^2})$, then it is a good idea to use this distribution as $f(y_0)$.
127
+
- Why? Because $y_0$ contains information about $\rho, \sigma_x$.
128
+
* If we suspect that $y_0$ is far in the tail of the stationary distribution -- so that variation in early observations in the sample has a significant *transient component* -- it is better to condition on $y_0$ by setting $f(y_0) = 1$.
139
129
140
130
To illustrate the issue, we'll begin by choosing an initial $y_0$ that is far out in the tail of the stationary distribution.
Evidently, the posteriors aren't centered on the true values of $.5, 1$ that we used to generate the data.
240
230
241
-
This is a symptom of the classic **Hurwicz bias** for first order autoregressive processes (see {cite}`hurwicz1950least`.)
231
+
This is a symptom of the classic **Hurwicz bias** for first order autoregressive processes (see {cite}`hurwicz1950least`.)
242
232
243
233
The Hurwicz bias is worse the smaller is the sample (see {cite}`Orcutt_Winokur_69`).
244
234
@@ -315,4 +305,4 @@ is telling `numpyro` to explain what it interprets as "explosive" observations e
315
305
316
306
Bayes' Law is able to generate a plausible likelihood for the first observation by driving $\rho \rightarrow 1$ and $\sigma \uparrow$ in order to raise the variance of the stationary distribution.
317
307
318
-
Our example illustrates the importance of what you assume about the distribution of initial conditions.
308
+
Our example illustrates the importance of what you assume about the distribution of initial conditions.
0 commit comments