Add unemployment dynamics lectures (linear + nonlinear)#928
Merged
Conversation
Implements issue #910 as a self-contained pair of "Data and Empirics" lectures on Bayesian time-series modeling of US unemployment, estimated with NUTS in NumPyro. Drops the fisheries throughline and the sinh cautionary tale from the original draft. unemployment_linear.md — A Linear Model of Unemployment - Random walk (mass escapes every bounded interval) -> linear AR(1). - The "is it a random walk?" question: monthly phi crowds against 1 (~9yr half-life), annual phi ~0.81; stationary spread and half-life. - Honest account of what's wrong with the linear model: near unit root, unbounded pull, constant reversion speed. - Exercises tie to ar1_turningpts: plug-in vs posterior-integrated predictive fan charts, and a Wecker-style path statistic (max unemployment over the next 8 years). - Distinct from ar1_bayes by design: real data, the random-walk question. unemployment_nonlinear.md — A Nonlinear Model of Unemployment - Motivated by the linear model's weaknesses; saturating tanh restoring force with a bounded pull, canonical form u_{t+1}=u_t+b*tanh(l(u_t-ubar))+e. - Dynamics: 45-degree/cobweb, the separate roles of beta and lambda, iso-(beta*lambda) "ridge before estimation", stationary distribution. - Identification contrast: monthly (lambda->0, beta-lambda ridge, ~random walk) vs annual (lambda identified, ridge dissolves). - Honest linear-vs-nonlinear verdict: fitted restoring forces coincide in the data-rich center and diverge only at recession extremes. Both verified end-to-end (NUTS, 4 chains, R-hat=1.0); added to _toc.yml under Data and Empirics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8 tasks
📖 Netlify Preview Ready!Preview URL: https://pr-928--sunny-cactus-210e3e.netlify.app Commit: 📚 Changed LecturesBuild Info
|
Connect the "is unemployment a random walk?" discussion to the natural-rate vs hysteresis debate of the 1980s-90s: - Overview now flags the debate (Friedman's natural rate vs Blanchard-Summers hysteresis), with the Nelson-Plosser irony that unemployment was their one stationary series. - A note in the phi-section explains why near-unit-root phi makes the debate hard to settle (low test power) and points to the nonlinear resolution pursued in unemployment_nonlinear. - Adds 5 references to _static/quant-econ.bib (Friedman 1968, Nelson-Plosser 1982, Blanchard-Summers 1986, Røed 1997, Kapetanios-Shin-Snell 2003). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…currence Replace the "linear pull is unbounded, which is bad" argument with a safer, positive observation: viewed linearly the data look like a random walk, yet unemployment stays in a band for decades. The linear model can reconcile these only on a knife-edge (phi just below 1); nonlinearity reconciles them structurally -- random-walk-like in normal times, with a firmer restoring force far from the natural rate that guarantees recurrence. - unemployment_linear: rename "What's unsatisfying..." to "Random walk, yet recurrent" and rewrite around the reconciliation; update the bridge prose under the scatter. Also incorporates John's Overview edits plus minor grammar fixes. - unemployment_nonlinear: reframe the Overview and conclusion to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add numbered MyST figures (mystnb caption + name) and {numref}
references throughout both lectures (4 in linear, 10 in nonlinear);
leave exercise-solution figures uncaptioned to avoid the LaTeX-float
PDF-build issue.
- Redesign the random-walk figure in the linear lecture: drop the 90%
band and observed line; mark the observed min/max with dashed lines
and shade the band between them.
- Rewrite the half-life discussion with a student-friendly derivation
(geometric/radioactive-style decay -> solve phi^k = 1/2).
- Shorten the linear lecture by cutting the "Random walk, yet recurrent"
section; the preceding note already bridges to unemployment_nonlinear
and the recurrence framing remains in that lecture's overview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts: # lectures/_static/quant-econ.bib
Reorganize the linear lecture into a tighter logical flow:
- start with monthly data only and fit the linear AR(1), working through
the mechanics and reading the posterior output (the "applying Bayesian
methods to real data" teaching section);
- find phi ~ 1 and connect it to the natural-rate vs hysteresis debate
(moved here from the Overview), including the Nelson-Plosser irony;
- ask whether it could be a unit root and give two reasons it is not:
(a) annual data show clear reversion -- framed as a statistical-power
point (a single linear AR(1) predicts phi_annual ~ phi_monthly^12), not
evidence of nonlinearity, with the half-life discussion deferred here to
emphasize robust annual reversion; (b) a literal random walk would
wander out of the historical band;
- reconcile: a linear model fits only on a knife-edge (phi indistinguishable
from one, constant reversion speed), which motivates the nonlinear model
in the sequel.
Single-panel monthly data plot; figures and {numref} references retained.
Verified end-to-end on GPU.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tighten the prose throughout (incorporates John's edits): trim the
overview and the monthly-fit commentary, drop the half-life subsection,
and shorten the reconciliation. Fixes: restore an accurate heading ("A
random walk would wander off" — a 1-D random walk is recurrent, not
transient) and three small typos/grammar slips (coefficient, accommodate,
"a random walk").
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the additive-tanh-restoring-force model with the S-shaped map
u_{t+1} = ubar + A*tanh(phi*(u-ubar)/A) + eps, which hugs the 45-degree
line through the normal band (random-walk-like) and flattens toward
asymptotes ubar +/- A outside it (bounded, recurrent). This ties the two
lectures together: phi is the same persistence as the linear model, and
the linear model is the A->infinity limit.
Rewrites every section (model, dynamics, estimation, linear-vs-nonlinear,
exercises). Estimation now shows monthly phi~1 with A unidentified
(consistent with linear) versus annual phi<1 with a finite ceiling, and a
note is honest that the growing pull at the extremes is a simplification.
Verified on GPU: figures render, fits converge (r_hat~1), PPC band stays
positive.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reframe the two-lecture sequence so the narrative is honest end-to-end. Lecture 1 (unemployment_linear): reorder around the historical argument. Lead with the natural-rate vs hysteresis debate, reject a literal random walk via the wander-off argument, then fit the AR(1) on [0,1) with the exclusion of 1 justified by that argument. Focus on the mechanics of fitting and the monthly-vs-annual contrast. Close with the asymmetry the symmetric Gaussian model cannot capture (heavy-tailed, right-skewed residuals), motivating the sequel. Lecture 2 (renamed unemployment_nonlinear -> unemployment_shocks, "Asymmetry and Large Shocks in Unemployment"): drop the S-shaped mean model, which cross-validation showed barely beat a straight line. Model the shocks instead: a linear reversion with a two-component (quiet/recession-jump) innovation. Reproduces the sawtooth and the right-skew of annual changes. Teach LOO model comparison in detail (jump beats linear by ~10 elpd, ~1.7 se) with its exchangeability caveat, plus a posterior predictive check on the asymmetry. Persistence is shown to be robust to the shock model, so the two questions are answered independently. Verified end-to-end on GPU: both lectures execute, fits converge (r_hat ~ 1), all figures and cross-references resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reorganize around estimate -> why it can't be a literal random walk (wander-off) -> corroborate with annual data. Fix the resulting forward reference in the Priors section, tidy the natural-rate/hysteresis history so the Nelson-Plosser irony is consistent with the body, and fix two small typos. Expand "What the linear model misses" into a slow, step-by-step explanation of the residual diagnostic: what the model assumes about the shocks, how we recover them by plugging in posterior medians (stated plainly), and how matching the overlaid Gaussian's variance isolates a difference in shape. Adds the skewness formula and a note that the full Bayesian version is the posterior predictive check in the sequel. Verified on GPU: executes, residual skewness 0.39 unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the model-comparison section of unemployment_shocks into a proper, self-contained treatment of Bayesian model comparison: the guiding principle (out-of-sample prediction, not in-sample fit), the log score and expected log predictive density, the leave-one-out estimate, and the importance-sampling trick that makes it computable from a single fit -- shown by hand (harmonic mean of the per-draw likelihoods) and matched against ArviZ's PSIS, with the Pareto-k diagnostic, p_loo, and the relation to AIC/BIC. Also bring the framing into line with where the two lectures ended up: elevate LOO from an aside to a stated payoff, add the two reusable tools (posterior predictive checks and cross-validation) to the conclusion, and tighten a redundant transition in the linear lecture. Verified on GPU: shocks lecture executes, by-hand LOO numbers (elpd -93.9 vs -105.7, difference 11.8 +/- 5.7) match az.loo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ation Shocks lecture: - Add a short "Accommodating the time series structure" subsection explaining leave-future-out CV, why it is the formally correct measure for a time series, and why it costs more (no importance-sampling shortcut); we ran it and it points the same way (verified on GPU). - Make the LOO derivation rigorous: spell out the importance-sampling steps from the leave-one-out posterior to the harmonic mean, and introduce S (the number of posterior draws) explicitly. - Rename "Scoring a prediction" -> "Leave-one-out cross-validation" to match its content; reinstate the elpd definition and a sentence on why the predictive scores are summed in logs; trim an unsupported persistence claim from the conclusion. Incorporates John's cuts. Linear lecture: incorporate John's cuts and reordering; fix a sentence fragment, a missing period, and a dash inconsistency. Both lectures convert and (where code changed earlier) execute on GPU. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…j anchor
Now that the (nuts)= anchor is on main (merged), switch the plain
{doc}`bayes_nonconj` NUTS link to {ref}`introduction to NUTS <nuts>`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged toolchain (myst v3.0.1) fails to register the LaTeX label for
the one wide 3-panel figure (fig-ppc), so {numref}`fig-ppc` raised
"undefined label" and failed the PDF build. Reference the panels directly
in the prose instead; the figure and its caption are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The model-comparison section imports arviz, but arviz is not in the build environment.yml — every lecture that uses it (ar1_bayes, ar1_turningpts) pip-installs it. Re-add the install cell that was wrongly removed; the HTML build re-executed the notebook and failed on ModuleNotFoundError. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #910 as a self-contained pair of lectures in the Data and Empirics section, on Bayesian time-series modeling of US unemployment (NUTS in NumPyro). Per discussion, this drops the original draft's fisheries throughline and
sinhcautionary tale, and refocuses on the dynamics of a one-dimensional model plus an honest account of what the data can and cannot support.Lectures
unemployment_linear.md— A Linear Model of Unemploymentar1_turningpts: plug-in vs. posterior-integrated predictive fan charts (the conditional-vs-extended distinction), and a Wecker-style path statistic (max unemployment over the next 8 years; P(reaches 7%) ≈ 0.32 from the end-2019 low).ar1_bayes: real data, and the random-walk question rather than the initial-condition focus.unemployment_nonlinear.md— A Nonlinear Model of Unemploymenttanhrestoring force with a bounded pull:Verification
Both lectures convert (
jupytext→py) and run end-to-end on a GPU (NUTS, 4 chains,chain_method="vectorized", R̂=1.0, 0 divergences); every figure was inspected. Added to_toc.ymlunder Data and Empirics.Notes for reviewers
{doc}links among these lectures and toar1_bayes,bayes_nonconj,ar1_turningpts).UNRATE) viapandas_datareader, consistent with existing lectures.🤖 Generated with Claude Code