@@ -98,12 +98,12 @@ sum of earnings
9898``` {math}
9999:label: obj_model
100100
101- {\mathbb E} \sum_{t=0}^\infty \beta^t u( y_t)
101+ {\mathbb E} \sum_{t=0}^\infty \beta^t y_t
102102```
103103
104104The constant $\beta$ lies in $(0, 1)$ and is called a ** discount factor** .
105105
106- The smaller is $\beta$, the more the agent discounts future utility relative to current utility .
106+ The smaller is $\beta$, the more the agent discounts future earnings relative to current earnings .
107107
108108The variable $y_t$ is income, equal to
109109
@@ -118,7 +118,7 @@ The worker faces a trade-off:
118118* Waiting too long for a good offer is costly, since the future is discounted.
119119* Accepting too early is costly, since better offers might arrive in the future.
120120
121- To decide optimally in the face of this trade-off, we use [ dynamic programming] ( https://dp.quantecon.org/ ) .
121+ To decide the optimal wait time in the face of this trade-off, we use [ dynamic programming] ( https://dp.quantecon.org/ ) .
122122
123123Dynamic programming can be thought of as a two-step procedure that
124124
@@ -137,25 +137,26 @@ In order to optimally trade-off current and future rewards, we need to think abo
137137To weigh these two aspects of the decision problem, we need to assign * values*
138138to states.
139139
140- To this end, let $v^* (w)$ be the total lifetime * value* accruing to an
140+ To this end, let $v^* (w)$ be the total lifetime value accruing to an
141141unemployed worker who enters the current period unemployed when the wage is
142142$w \in \mathbb{W}$.
143143
144144(In particular, the agent has wage offer $w$ in hand and can accept or reject it.)
145145
146- More precisely, $v^* (w)$ denotes the value of the objective function
147- {eq}` obj_model ` when an agent in this situation makes * optimal* decisions now
148- and at all future points in time.
146+ More precisely, $v^* (w)$ denotes the total sum of expected discounted earnings
147+ when an agent always behaves in an optimal way. points in time.
149148
150149Of course $v^* (w)$ is not trivial to calculate because we don't yet know
151150what decisions are optimal and what aren't!
152151
153- But think of $v^* $ as a function that assigns to each possible wage
154- $s$ the maximal lifetime value that can be obtained with that offer in
155- hand.
152+ If we don't know what opimal choices are, it feels imposible to calculate
153+ $v^* (w)$.
156154
157- A crucial observation is that this function $v^* $ must satisfy the
158- recursion
155+ But let's put this aside for now and think of $v^* $ as a function that assigns
156+ to each possible wage $w$ the maximal lifetime value $v^* (w)$ that can be
157+ obtained with that offer in hand.
158+
159+ A crucial observation is that this function $v^* $ must satisfy
159160
160161``` {math}
161162:label: odu_pv
@@ -175,6 +176,7 @@ ubiquitous in economic dynamics and other fields involving planning over time.
175176The intuition behind it is as follows:
176177
177178* the first term inside the max operation is the lifetime payoff from accepting current offer, since
179+ such a worker works forever at $w$ and values this income stream as
178180
179181$$
180182 \frac{w}{1 - \beta} = w + \beta w + \beta^2 w + \cdots
@@ -189,16 +191,24 @@ lifetime value from today, given current offer $w$.
189191
190192But this is precisely $v^* (w)$, which is the left-hand side of {eq}` odu_pv ` .
191193
194+ Putting this all together, we see that {eq}` odu_pv ` is valid for all $w$.
195+
192196
193197### The Optimal Policy
194198
195- Suppose for now that we are able to solve {eq}` odu_pv ` for the unknown function $v^* $.
199+ We still don't know how to compute $v^* $ (although {eq}` odu_pv ` gives us hints
200+ we'll return to below).
201+
202+ But suppose for now that we do know $v^* $.
196203
197- Once we have this function in hand we can behave optimally (i.e., make the
198- right choice between accept and reject).
204+ Once we have this function in hand we can easily make optimal choices (i.e., make the
205+ right choice between accept and reject given any $w$ ).
199206
200207All we have to do is select the maximal choice on the right-hand side of {eq}` odu_pv ` .
201208
209+ In other words, we make the best choice between stopping and continuing, given
210+ the information provided to us by $v^* $.
211+
202212The optimal action is best thought of as a ** policy** , which is, in general, a map from
203213states to actions.
204214
@@ -262,7 +272,7 @@ In view of {eq}`odu_pv`, this vector satisfies the nonlinear system of equations
262272
263273v^*(i)
264274= \max \left\{
265- \frac{w(i)}{1 - \beta}, \, c + \beta \sum_{1 \leq j \leq n}
275+ \frac{w(i)}{1 - \beta}, \, c + \beta \sum_{j=1}^n
266276 v^*(j) q (j)
267277 \right\}
268278\quad
@@ -284,7 +294,7 @@ Step 2: compute a new vector $v' \in \mathbb R^n$ via
284294
285295v'(i)
286296= \max \left\{
287- \frac{w(i)}{1 - \beta}, \, c + \beta \sum_{1 \leq j \leq n}
297+ \frac{w(i)}{1 - \beta}, \, c + \beta \sum_{j=1}^n
288298 v(j) q (j)
289299 \right\}
290300\quad
@@ -312,7 +322,7 @@ First, one defines a mapping $T$ from $\mathbb R^n$ to itself via
312322
313323(Tv)(i)
314324= \max \left\{
315- \frac{w(i)}{1 - \beta}, \, c + \beta \sum_{1 \leq j \leq n}
325+ \frac{w(i)}{1 - \beta}, \, c + \beta \sum_{j=1}^n
316326 v(j) q (j)
317327 \right\}
318328\quad
@@ -341,8 +351,7 @@ $\{ T^k v \}$ converges to the fixed point $v^*$ regardless of $v$.
341351
342352### Implementation
343353
344- Our default for $q$, the distribution of the state process, will be
345- [ Beta-binomial] ( https://en.wikipedia.org/wiki/Beta-binomial_distribution ) .
354+ Our default for $q$, the wage offer distribution, will be [ Beta-binomial] ( https://en.wikipedia.org/wiki/Beta-binomial_distribution ) .
346355
347356``` {code-cell} ipython3
348357n, a, b = 50, 200, 100 # default parameters
@@ -381,7 +390,23 @@ class McCallModel(NamedTuple):
381390 q: jnp.ndarray = q_default # array of probabilities
382391```
383392
384- We implement the Bellman operator $T$ from {eq}` odu_pv3 ` as follows
393+ We implement the Bellman operator $T$ from {eq}` odu_pv3 ` , which we can write in
394+ terms of array operations as
395+
396+ ``` {math}
397+ :label: odu_pv4
398+
399+ Tv
400+ = \max \left\{
401+ \frac{w}{1 - \beta}, \, c + \beta \sum_{j=1}^n v(j) q (j)
402+ \right\}
403+ \quad
404+ ```
405+
406+ (The first term inside the max is an array and the second is just a number -- here
407+ we mean that the max comparison against this number is done element-by-element for all elements in the array.)
408+
409+ We can code $T$ up as follows.
385410
386411``` {code-cell} ipython3
387412def T(model: McCallModel, v: jnp.ndarray):
@@ -416,7 +441,7 @@ plt.show()
416441You can see that convergence is occurring: successive iterates are getting closer together.
417442
418443Here's a more serious iteration effort to compute the limit, which continues
419- until measured deviation between successive iterates is below tol.
444+ until measured deviation between successive iterates is below ` tol ` .
420445
421446Once we obtain a good approximation to the limit, we will use it to calculate
422447the reservation wage.
@@ -459,8 +484,42 @@ print(res_wage)
459484Now that we know how to compute the reservation wage, let's see how it varies with
460485parameters.
461486
462- In particular, let's look at what happens when we change $\beta$ and
463- $c$.
487+ ``` {code-cell} ipython3
488+ # Plot wage distribution with reservation wages before and after changing beta
489+ fig, ax = plt.subplots()
490+
491+ # Get the default color cycle
492+ prop_cycle = plt.rcParams['axes.prop_cycle']
493+ colors = prop_cycle.by_key()['color']
494+
495+ # Plot the wage offer distribution
496+ ax.plot(w, q, '-o', alpha=0.6, label='wage offer distribution', color=colors[0])
497+
498+ # Compute reservation wage with default beta
499+ model_default = McCallModel()
500+ v_init = model_default.w / (1 - model_default.β)
501+ v_default, res_wage_default = compute_reservation_wage(model_default, v_init)
502+
503+ # Compute reservation wage with lower beta
504+ β_new = 0.96
505+ model_low_beta = McCallModel(β=β_new)
506+ v_init_low = model_low_beta.w / (1 - model_low_beta.β)
507+ v_low, res_wage_low = compute_reservation_wage(model_low_beta, v_init_low)
508+
509+ # Plot vertical lines for reservation wages
510+ ax.axvline(x=res_wage_default, color=colors[1], lw=2,
511+ label=f'reservation wage (β={model_default.β})')
512+ ax.axvline(x=res_wage_low, color=colors[2], lw=2,
513+ label=f'reservation wage (β={β_new})')
514+
515+ ax.set_xlabel('wage', fontsize=12)
516+ ax.set_ylabel('probability', fontsize=12)
517+ ax.tick_params(axis='both', which='major', labelsize=11)
518+ ax.legend(loc='upper left', frameon=False, fontsize=11)
519+ plt.show()
520+ ```
521+
522+ Now let's look more systematically at what happens when we change $\beta$ and $c$.
464523
465524As a first step, given that we'll use it many times, let's create a more
466525efficient, jit-complied version of the function that computes the reservation
0 commit comments