From acaa544e591d0e9bbdad0ca8682ad2f01cec07d7 Mon Sep 17 00:00:00 2001 From: igerber Date: Mon, 29 Jun 2026 17:20:25 -0400 Subject: [PATCH 1/3] test(lpdid): non-absorbing R-parity via independent feols Eq.12/13, Phase C2 Pin the C1 non-absorbing modes (non_absorbing="first_entry"/"effect_stabilization") against an INDEPENDENT fixest::feols reconstruction of the paper's Eq. 12 / Eq. 13 clean-sample restrictions (explicit per-window indexing, derived from the paper). Variance-weighted point and SE match the library to ~1e-13/~1e-15; the reweighted point matches and its SE is pinned as a regression guard (a small weighted-cluster convention difference vs feols). The recipe's independence was demonstrated when an earlier draft's Eq. 12 control off-by-one diverged from the already-correct library and was corrected against the paper, plus a hand-computed Python micro-check. alexCardazzi/lpdid's nonabsorbing_lag is NOT a faithful Eq. 13 (it clamps treat_diff[<0]<-0 so its clean-control window blocks only turn-ons; it reuses a forward placebo window; it NA-excludes pre-panel-treated rows where the library clamps pre-min_t to untreated). It diverges ~0.01-0.05 from Eq. 13 even on a monotone no-off-switch panel, so it is recorded in the golden meta as a divergent third-party reference, not a parity gate (the alexCardazzi-pooled precedent). Verify-and-document PR, no estimator change. Separate non-absorbing panel + golden keep the B2 absorbing goldens byte-identical (verified). Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 10 + TODO.md | 2 +- benchmarks/R/generate_lpdid_golden.R | 171 ++++ .../data/lpdid_nonabsorbing_golden.json | 55 ++ benchmarks/data/lpdid_nonabsorbing_panel.csv | 841 ++++++++++++++++++ docs/methodology/REGISTRY.md | 5 +- tests/test_methodology_lpdid.py | 123 +++ 7 files changed, 1204 insertions(+), 3 deletions(-) create mode 100644 benchmarks/data/lpdid_nonabsorbing_golden.json create mode 100644 benchmarks/data/lpdid_nonabsorbing_panel.csv diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f7ff8b5..31e14293 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **`LPDiD` non-absorbing R-parity validation** (Phase C2). Pins both non-absorbing modes + against an independent `fixest::feols` reconstruction of the paper's Eq. 12 (`first_entry`) + and Eq. 13 (`effect_stabilization`) clean-sample restrictions: variance-weighted point and + SE match to ~1e-13/~1e-15; the `effect_stabilization` reweighted point matches (its SE is + pinned as a regression guard, a small weighted-cluster convention difference). New `tests/test_methodology_lpdid.py` + parity class + separate `lpdid_nonabsorbing_panel.csv` / `lpdid_nonabsorbing_golden.json` + (the absorbing B2 goldens stay byte-identical). `alexCardazzi/lpdid`'s `nonabsorbing_lag` is + recorded as a divergent third-party reference (it clamps off-switches and uses a non-paper + boundary/placebo convention, diverging ~0.01-0.05 from Eq. 13 even on a monotone panel), not + a parity gate. No estimator change. - **`LPDiD` non-absorbing (reversible) treatment** (Dube, Girardi, Jordà & Taylor 2025, Section 4.2). New `non_absorbing` parameter: `"first_entry"` (Eq. 12 — the effect of entering treatment for the first time and staying treated) and `"effect_stabilization"` diff --git a/TODO.md b/TODO.md index 9be6b18c..ff84963b 100644 --- a/TODO.md +++ b/TODO.md @@ -118,7 +118,7 @@ exists but parity can't be verified without a local toolchain. | **`bias_corrected_local_linear` (lprobust) Phase-1c follow-ups:** extend golden parity to `kernel ∈ {triangular, uniform}` (epa-only today); expose `vce ∈ {hc0,hc1,hc2,hc3}` on the public wrapper once R goldens exist (port supports all four; needs a per-mode generator + a hc2/hc3 q-fit-leverage decision); clustered-DGP auto-bandwidth parity is **blocked upstream** on an nprobust singleton-cluster bug in `lpbwselect.mse.dpi` (Phase-1c DGP 4 uses manual `h=b=0.3`). | `_nprobust_port.py`, `local_linear.py`, `generate_nprobust_lprobust_golden.R` | Phase 1c | Low-Med | | `HeterogeneousAdoptionDiD` Stute-family Stata-bridge parity: no public R `Stutetest` package exists; would add `benchmarks/stata/generate_stute_golden.do` + a Stata dependency. | `benchmarks/stata/`, `tests/test_stute_test_parity.py` | follow-up | Low | | **`LPDiD` regression-adjustment SE — no runnable R reference.** The RA influence-function cluster SE is canonically Stata `teffects ra ... atet vce(cluster)` only; no R package computes it (`alexCardazzi/lpdid` does direct covariate inclusion, not RA). Today the RA *point* is R-anchored (~1e-12), the SE is pinned + MC-coverage-validated (`coverage_lpdid_ra.py`). Follow-up: contribute the RA path to `alexCardazzi/lpdid` so a runnable R RA reference exists — only a *trusted* anchor once cross-checked vs Stata `teffects` (else circular). | `tests/test_methodology_lpdid.py`, `benchmarks/python/coverage_lpdid_ra.py` | #B2 follow-up | Low | -| **`LPDiD` non-absorbing R-parity (PR-C2).** The shipped non-absorbing modes (`first_entry` Eq. 12 / `effect_stabilization` Eq. 13) are validated by pure-Python tests (absorbing reduction, re-entry mechanism, placebo, non-negative weighting, DGP recovery) but not yet pinned to `alexCardazzi/lpdid`'s `nonabsorbing` / `nonabsorbing_lag` branches. PR-C2: smoke-gate the option mapping (incl. the equal-weight reweight and the pre-observation boundary convention), then extend `generate_lpdid_golden.R` + `test_methodology_lpdid.py` with non-absorbing variants. | `benchmarks/R/generate_lpdid_golden.R`, `tests/test_methodology_lpdid.py` | PR-C2 | Medium | +| **`LPDiD` non-absorbing R-parity - DONE (PR-C2)** via an independent `fixest::feols` Eq. 12/13 reconstruction (point+SE ~1e-13/~1e-15 vw; `effect_stabilization` reweighted point + pinned SE). `alexCardazzi/lpdid`'s `nonabsorbing_lag` proved NOT a faithful Eq. 13 (off-switch clamp + non-paper boundary/placebo window; diverges ~0.01-0.05 even on a monotone panel), so it is recorded as a divergent reference, not a gate. **Residual external-reference gap:** the authors' canonical non-absorbing SE/RA is Stata `lpdid`/`teffects` only (no faithful R analogue) - same class as the absorbing RA-SE row above; revisit if a Stata toolchain or a corrected R package appears. | `benchmarks/R/generate_lpdid_golden.R`, `tests/test_methodology_lpdid.py` | PR-C2 | Low | | `HeterogeneousAdoptionDiD` Phase-3 R-parity: ships coverage-rate validation on synthetic DGPs, not tight point parity vs `chaisemartin::stute_test` / `yatchew_test` (needs bootstrap-seed-semantics + `B` alignment across numpy/R). | `tests/test_had_pretests.py` | Phase 3 | Low | ### Parked — pending user demand / out of scope diff --git a/benchmarks/R/generate_lpdid_golden.R b/benchmarks/R/generate_lpdid_golden.R index 25417f0a..d95b9c27 100644 --- a/benchmarks/R/generate_lpdid_golden.R +++ b/benchmarks/R/generate_lpdid_golden.R @@ -312,3 +312,174 @@ golden_path <- file.path("benchmarks", "data", "lpdid_golden.json") # Python loader reads as None and handles in its missing-value branch. write_json(golden, golden_path, auto_unbox = TRUE, pretty = TRUE, digits = 12, na = "null") message(sprintf("Wrote golden: %s", golden_path)) + +# ============================================================ +# 5. NON-ABSORBING (Phase C2): independent feols reconstruction of the paper's +# Eq. 12 (first_entry) and Eq. 13 (effect_stabilization) clean-sample +# restrictions, anchoring the C1 non-absorbing modes (LPDiD(non_absorbing=...)). +# EXPLICIT per-window indexing (a different construction than the Python +# cumulative-lookup; the recipe's independence was demonstrated when an earlier +# draft's Eq.12 control off-by-one diverged from the already-correct Python and +# was corrected against the paper). Validated == the library to ~1e-13 on point +# and ~1e-15 on SE for the variance-weighted variants (reweighted: point ~1e-13, +# SE has a small feols-weighted-cluster convention difference, pinned on the +# Python side). alexCardazzi::lpdid()'s `nonabsorbing_lag` is NOT a faithful +# Eq.13 (it clamps off-switches via treat_diff[<0]<-0 and uses a non-paper +# boundary/window convention; it diverges ~0.01-0.05 from Eq.13 even on a +# monotone no-off-switch panel) -> recorded in `meta` as a divergent third-party +# reference, NOT a parity gate (the B2 alexCardazzi-pooled precedent). +# +# APPENDED after the absorbing write_json with its OWN set.seed and distinct +# object names; setFixest_ssc is NOT re-called -> the absorbing panel + golden +# above are byte-identical. +# ============================================================ +NA_L <- 3L; NA_PRE <- 3L; NA_POST <- 4L; NA_TT <- 14L +set.seed(424242) # isolated; only the rnorm() calls below consume this stream + +na_mk <- function(uid, dpath) { + ufe <- rnorm(1, 0, 1.5) + data.table(unit = uid, time = 1:NA_TT, treat = as.integer(dpath), + y = ufe + 0.4 * (1:NA_TT) + 2.0 * as.integer(dpath) + rnorm(NA_TT, 0, 0.3)) +} +na_rows <- list(); na_uid <- 0L +na_add <- function(n, gen) for (i in 1:n) { na_uid <<- na_uid + 1L; na_rows[[length(na_rows)+1]] <<- na_mk(na_uid, gen()) } +na_add(18, function() as.integer((1:NA_TT) >= sample(4:8, 1))) # enter & stay +na_add(12, function() { g <- sample(3:6, 1); ex <- g + sample(2:3, 1); re <- ex + sample(2:4, 1) + d <- as.integer((1:NA_TT) >= g); d[(1:NA_TT) >= ex] <- 0L; if (re <= NA_TT) d[(1:NA_TT) >= re] <- 1L; d }) # reversal / re-entry +na_add(10, function() rep(1L, NA_TT)) # stabilized treated (eq13 controls) +na_add(20, function() rep(0L, NA_TT)) # never treated +na_dt <- rbindlist(na_rows); setorder(na_dt, unit, time) +na_panel_path <- file.path("benchmarks", "data", "lpdid_nonabsorbing_panel.csv") +fwrite(na_dt, na_panel_path) +message(sprintf("Wrote non-absorbing panel: %s (%d rows, %d units)", + na_panel_path, nrow(na_dt), uniqueN(na_dt$unit))) + +# ---- explicit per-window mask helpers (independent of the Python) ---- +# Each unit is gap-free over 1:NA_TT; positions < 1 are treated as D=0 (the +# "untreated before the first observed period" convention - this is where alex's +# NA-exclude differs from the library's clamp-to-untreated). +na_Dwin <- function(D, a, b) { a <- max(a, 1L); if (b < a) integer(0) else D[a:b] } +na_ddwin <- function(dd, a, b) { a <- max(a, 1L); if (b < a) integer(0) else dd[a:b] } +na_clean_sample <- function(dt, mode, h) { + dt[, { + D <- treat; n <- length(D) + dd <- D - c(0L, D[-n]) # first diff; pre-series untreated -> dd[1]=D[1] + cumD_before <- cumsum(c(0L, D))[1:n] # D summed strictly before each position + cumD_incl <- cumsum(D) # D summed THROUGH each position (inclusive) + cdi <- function(k) if (k < 1L) 0L else cumD_incl[min(k, n)] + role <- rep(NA_character_, n) + for (t in 1:n) { + tgt <- t + h; base <- t - 1L + if (tgt < 1L || tgt > n || base < 1L) next # need baseline y_{t-1} and target y_{t+h} + if (mode == "eq13") { + if (h >= 0L) { + treated <- dd[t] == 1L && all(na_Dwin(D, t - NA_L, t - 1L) == 0L) && + all(na_ddwin(dd, t + 1L, t + h) == 0L) # entry, D=0 on [t-L,t-1], no other change in (t,t+h] + control <- all(na_ddwin(dd, t - NA_L, t + h) == 0L) # no treatment change in [t-L,t+h] + } else { + m <- max(NA_L, -h) + treated <- dd[t] == 1L && all(na_Dwin(D, t - m, t - 1L) == 0L) + control <- all(na_ddwin(dd, t - m, t - 1L) == 0L) # placebo window backward to t+h + } + } else { # eq12 first_entry + entry <- dd[t] == 1L && cumD_before[t] == 0L # first onset (D=0 strictly before t) + if (h >= 0L) { + treated <- entry && all(na_ddwin(dd, t + 1L, t + h) == 0L) # stays treated through t+h + control <- cdi(t + h) == 0L # untreated through t+h INCLUSIVE (Eq.12) + } else { + treated <- entry + control <- cdi(t) == 0L + } + } + if (isTRUE(treated)) role[t] <- "T" else if (isTRUE(control)) role[t] <- "C" + } + .(time = time, role = role) + }, by = unit][!is.na(role)] +} +na_es_one <- function(h, reweight = FALSE) { + cs <- na_clean_sample(na_dt, na_mode, h) + if (nrow(cs) == 0L) return(c(NA_real_, NA_real_)) + d <- merge(na_dt, cs, by = c("unit", "time")) + d[, `:=`(tb = time - 1L, tt = time + h)] + d <- merge(d, na_dt[, .(unit, tb = time, yb = y)], by = c("unit", "tb")) + d <- merge(d, na_dt[, .(unit, tt = time, yt = y)], by = c("unit", "tt")) + d[, Dy := yt - yb]; d[, tr := as.integer(role == "T")] + d[, has_c := any(tr == 0L), by = time]; d <- d[has_c == TRUE] # drop event-times with no clean control + if (uniqueN(d$tr) < 2L) return(c(NA_real_, NA_real_)) + if (reweight) { # per-event-time N/N_control inverse weights + w <- d[, .(N = .N, nc = sum(tr == 0L)), by = time][, w := N / nc] + d <- merge(d, w[, .(time, w)], by = "time") + m <- feols(Dy ~ tr | time, data = d, weights = ~w, vcov = ~unit, warn = FALSE, notes = FALSE) + } else { + m <- feols(Dy ~ tr | time, data = d, vcov = ~unit, warn = FALSE, notes = FALSE) + } + c(unname(coef(m)["tr"]), unname(se(m)["tr"])) +} +na_es <- function(mode, reweight = FALSE) { + na_mode <<- mode + out <- list() + for (h in 0:NA_POST) out[[as.character(h)]] <- na_es_one(h, reweight) + for (h in 2:NA_PRE) out[[as.character(-h)]] <- na_es_one(-h, reweight) + out +} + +first_entry_es <- na_es("eq12") +effect_stab_es <- na_es("eq13") +effect_stab_rw_es <- na_es("eq13", reweight = TRUE) + +# ---- alexCardazzi nonabsorbing_lag: divergent reference (recorded, NOT gated) ---- +na_alex <- tryCatch({ + a <- lpdid::lpdid(as.data.frame(na_dt), window = c(-NA_PRE, NA_POST), y = "y", + unit = "unit", time = "time", treat_status = "treat", + cluster = "unit", nonabsorbing_lag = NA_L) + hs <- (-NA_PRE):NA_POST # alex coeftable rows map to horizons by position + setNames(lapply(seq_along(hs), function(i) + c(a$coeftable[i, "Estimate"], a$coeftable[i, "Std. Error"])), as.character(hs)) +}, error = function(e) list(error = conditionMessage(e))) + +na_golden <- list( + meta = list( + estimator = "LPDiD (Dube, Girardi, Jorda & Taylor 2025) - non-absorbing (Phase C2)", + r_version = R.version.string, + fixest_version = as.character(packageVersion("fixest")), + lpdid_alexcardazzi_version = as.character(packageVersion("lpdid")), + lpdid_alexcardazzi_commit = ALEX_SHA, + seed = 424242L, L = NA_L, pre_window = NA_PRE, post_window = NA_POST, + anchor = paste( + "Both modes are anchored on an INDEPENDENT feols reconstruction of the paper's", + "Eq. 12 (first_entry) and Eq. 13 (effect_stabilization) clean-sample restrictions", + "(explicit per-window indexing). Validated == the library to ~1e-13 (point) and", + "~1e-15 (SE) for the variance-weighted variants."), + reweight_se_note = paste( + "effect_stab_rw: POINT matches the library to ~1e-13; the reweighted SE has a", + "small feols-weighted-cluster convention difference (~5e-5) vs the library, so the", + "library reweighted SE is pinned as a regression guard on the Python side, not", + "asserted against feols."), + first_entry_note = paste( + "first_entry (Eq. 12) has NO R-package analogue (alexCardazzi only exposes the", + "effect-stabilization nonabsorbing_lag). It is anchored on the independent feols", + "Eq. 12 recipe + a hand-computed Python micro-check. Its absorbing reduction", + "(first_entry == absorbing on absorbing panels) is a separate internal check, NOT", + "an R anchor for non-absorbing behavior."), + alex_note = paste( + "alexCardazzi::lpdid() `nonabsorbing_lag` is NOT a faithful paper Eq. 13: it clamps", + "treat_diff[<0]<-0 (its clean-control window blocks only treatment turn-ONS, so a", + "unit turning OFF inside [t-L,t+h] still counts as a control) and reuses a forward", + "window for placebos. The library uses the literal 'no treatment change' (both", + "directions) + the backward placebo window [t-max(L,-h),t-1] (more paper-faithful).", + "alex diverges ~0.01-0.05 from Eq. 13 even on a monotone no-off-switch panel, so it", + "is recorded below for transparency only, NOT used as a parity gate."), + boundary_note = paste( + "Boundary convention (closes the REGISTRY 'confirm vs R in PR-C2' item): periods", + "before a unit's first observed period are treated as untreated/no-change (the", + "library clamps pre-min_t to 0); alexCardazzi NA-excludes such rows (first-row lag", + "is NA). A documented divergence, not a defect.") + ), + first_entry = first_entry_es, + effect_stab = effect_stab_es, + effect_stab_rw = effect_stab_rw_es, + alex_nonabsorbing_lag = na_alex +) +na_golden_path <- file.path("benchmarks", "data", "lpdid_nonabsorbing_golden.json") +write_json(na_golden, na_golden_path, auto_unbox = TRUE, pretty = TRUE, digits = 12, na = "null") +message(sprintf("Wrote non-absorbing golden: %s", na_golden_path)) diff --git a/benchmarks/data/lpdid_nonabsorbing_golden.json b/benchmarks/data/lpdid_nonabsorbing_golden.json new file mode 100644 index 00000000..02fa9d22 --- /dev/null +++ b/benchmarks/data/lpdid_nonabsorbing_golden.json @@ -0,0 +1,55 @@ +{ + "meta": { + "estimator": "LPDiD (Dube, Girardi, Jorda & Taylor 2025) - non-absorbing (Phase C2)", + "r_version": "R version 4.5.2 (2025-10-31)", + "fixest_version": "0.13.2", + "lpdid_alexcardazzi_version": "0.6.1", + "lpdid_alexcardazzi_commit": "ba64563983861be5e616f6020842c1a1cdf17a27", + "seed": 424242, + "L": 3, + "pre_window": 3, + "post_window": 4, + "anchor": "Both modes are anchored on an INDEPENDENT feols reconstruction of the paper's Eq. 12 (first_entry) and Eq. 13 (effect_stabilization) clean-sample restrictions (explicit per-window indexing). Validated == the library to ~1e-13 (point) and ~1e-15 (SE) for the variance-weighted variants.", + "reweight_se_note": "effect_stab_rw: POINT matches the library to ~1e-13; the reweighted SE has a small feols-weighted-cluster convention difference (~5e-5) vs the library, so the library reweighted SE is pinned as a regression guard on the Python side, not asserted against feols.", + "first_entry_note": "first_entry (Eq. 12) has NO R-package analogue (alexCardazzi only exposes the effect-stabilization nonabsorbing_lag). It is anchored on the independent feols Eq. 12 recipe + a hand-computed Python micro-check. Its absorbing reduction (first_entry == absorbing on absorbing panels) is a separate internal check, NOT an R anchor for non-absorbing behavior.", + "alex_note": "alexCardazzi::lpdid() `nonabsorbing_lag` is NOT a faithful paper Eq. 13: it clamps treat_diff[<0]<-0 (its clean-control window blocks only treatment turn-ONS, so a unit turning OFF inside [t-L,t+h] still counts as a control) and reuses a forward window for placebos. The library uses the literal 'no treatment change' (both directions) + the backward placebo window [t-max(L,-h),t-1] (more paper-faithful). alex diverges ~0.01-0.05 from Eq. 13 even on a monotone no-off-switch panel, so it is recorded below for transparency only, NOT used as a parity gate.", + "boundary_note": "Boundary convention (closes the REGISTRY 'confirm vs R in PR-C2' item): periods before a unit's first observed period are treated as untreated/no-change (the library clamps pre-min_t to 0); alexCardazzi NA-excludes such rows (first-row lag is NA). A documented divergence, not a defect." + }, + "first_entry": { + "0": [1.846815337457, 0.09153686444445], + "1": [1.917418918492, 0.08251475771707], + "2": [1.954156237069, 0.09599599361364], + "3": [1.843000209785, 0.09321550686961], + "4": [1.865104589049, 0.103446008623], + "-2": [-0.02850800971675, 0.05995016486332], + "-3": [-0.1105868819491, 0.08386469485401] + }, + "effect_stab": { + "0": [1.851991243051, 0.07114522249739], + "1": [1.921104469807, 0.06702437575657], + "2": [1.929898604367, 0.07968483555297], + "3": [1.913731884109, 0.07652544247632], + "4": [1.886119926793, 0.09228264962342], + "-2": [-0.08778777268896, 0.04731311562096], + "-3": [-0.1105312033519, 0.06317647498563] + }, + "effect_stab_rw": { + "0": [1.854916933798, 0.07249623837132], + "1": [1.919465194082, 0.06780622997428], + "2": [1.933380912538, 0.08001315758582], + "3": [1.915912905771, 0.07730903341075], + "4": [1.886338871188, 0.09301892258318], + "-2": [-0.08271702126404, 0.0478189929193], + "-3": [-0.1074628208957, 0.06454215400419] + }, + "alex_nonabsorbing_lag": { + "-3": [-0.145088679015, 0.09624515089736], + "-2": [-0.03709433061648, 0.06947456423817], + "-1": [0, 0], + "0": [1.884378366597, 0.1046700491971], + "1": [1.910655279694, 0.0935622160941], + "2": [1.517560001248, 0.1769881246961], + "3": [1.231229422875, 0.1947738412434], + "4": [1.358589829451, 0.1796027407839] + } +} diff --git a/benchmarks/data/lpdid_nonabsorbing_panel.csv b/benchmarks/data/lpdid_nonabsorbing_panel.csv new file mode 100644 index 00000000..2eead798 --- /dev/null +++ b/benchmarks/data/lpdid_nonabsorbing_panel.csv @@ -0,0 +1,841 @@ +unit,time,treat,y +1,1,0,0.89732222849164 +1,2,0,1.12761890238898 +1,3,0,1.84827553945422 +1,4,0,1.98225973822886 +1,5,0,2.60953144440623 +1,6,0,3.29316761081967 +1,7,1,5.68716086080081 +1,8,1,5.45347478945765 +1,9,1,6.23572471728418 +1,10,1,6.38754390631909 +1,11,1,6.86933253983245 +1,12,1,7.66453665076445 +1,13,1,8.17831620606333 +1,14,1,8.50258456854603 +2,1,0,1.48534422921369 +2,2,0,2.10959080256958 +2,3,0,2.04544451640549 +2,4,0,3.06735129979493 +2,5,0,3.60011506144951 +2,6,1,5.93565261117029 +2,7,1,6.51588506969162 +2,8,1,6.24218469629243 +2,9,1,6.48450760326006 +2,10,1,6.96426507837867 +2,11,1,7.96287896422567 +2,12,1,8.24219182235279 +2,13,1,8.18981773065376 +2,14,1,8.89721615001002 +3,1,0,0.393688348740403 +3,2,0,0.677253359643488 +3,3,0,1.52189623834243 +3,4,0,1.81161220807762 +3,5,1,4.30545814406444 +3,6,1,4.93802570421976 +3,7,1,5.08084962384247 +3,8,1,5.55475313004814 +3,9,1,5.50093318134781 +3,10,1,6.01101882766784 +3,11,1,6.20805085538213 +3,12,1,7.10912094146175 +3,13,1,7.7957194277904 +3,14,1,7.67613195775524 +4,1,0,1.5224733113123 +4,2,0,2.2017078757004 +4,3,0,3.05207298458609 +4,4,0,2.6828090659984 +4,5,0,3.60434791944179 +4,6,0,3.88006906242433 +4,7,1,6.05329596629539 +4,8,1,7.0156089429874 +4,9,1,7.45127565258193 +4,10,1,7.43134251502 +4,11,1,7.79781161597347 +4,12,1,7.85802233945652 +4,13,1,8.58324354293025 +4,14,1,9.72284149948647 +5,1,0,0.541086794030469 +5,2,0,0.173673619858856 +5,3,0,1.37371213234537 +5,4,0,1.52238933314589 +5,5,0,2.46270044302701 +5,6,1,3.22996188327354 +5,7,1,4.39567341954112 +5,8,1,4.56561829994327 +5,9,1,5.61296580998804 +5,10,1,5.74692678898234 +5,11,1,6.15428729593554 +5,12,1,6.54116445574898 +5,13,1,7.21832674970186 +5,14,1,7.41428096955636 +6,1,0,1.7042959217355 +6,2,0,1.36693177203776 +6,3,0,1.9213372657943 +6,4,0,2.27614840233565 +6,5,0,3.11600410334839 +6,6,0,3.85650173575443 +6,7,0,3.70758396584758 +6,8,1,6.03174156494841 +6,9,1,6.90395592285634 +6,10,1,7.36844158454031 +6,11,1,7.34130044100731 +6,12,1,8.40352365611302 +6,13,1,7.7161615855449 +6,14,1,8.67951739104569 +7,1,0,-0.184724880283169 +7,2,0,0.571387183193669 +7,3,0,0.878460478852466 +7,4,0,1.51566274866051 +7,5,1,3.71215839699409 +7,6,1,4.23138452949599 +7,7,1,4.92247147657239 +7,8,1,5.23247677879693 +7,9,1,4.66308504501579 +7,10,1,5.64154065769172 +7,11,1,5.67029957818338 +7,12,1,5.57619605790074 +7,13,1,6.31619942768947 +7,14,1,6.22685926109263 +8,1,0,-1.95203327881827 +8,2,0,-1.1398988763717 +8,3,0,-0.261694513940642 +8,4,0,0.14327991626873 +8,5,1,2.18395103669497 +8,6,1,2.11792829637893 +8,7,1,3.26484438808608 +8,8,1,3.41162054917584 +8,9,1,3.72923209901386 +8,10,1,4.6493088528185 +8,11,1,4.36040135070462 +8,12,1,5.19266213803402 +8,13,1,5.6022492854942 +8,14,1,6.16765388992712 +9,1,0,1.41051223155901 +9,2,0,1.91361580828875 +9,3,0,2.2520812848843 +9,4,0,2.61222012454907 +9,5,0,2.61646057594501 +9,6,1,4.70666178036149 +9,7,1,5.5462560300548 +9,8,1,5.39594810965109 +9,9,1,6.48668223941554 +9,10,1,6.65857952172302 +9,11,1,7.32502731152341 +9,12,1,7.6502839565631 +9,13,1,8.07652418995145 +9,14,1,8.65626253695282 +10,1,0,1.86153562576611 +10,2,0,2.33780058119514 +10,3,0,2.55981517688618 +10,4,0,2.78668889076721 +10,5,1,5.12767998324962 +10,6,1,5.76897835065833 +10,7,1,6.60899764814853 +10,8,1,6.52976922289668 +10,9,1,6.74891237704215 +10,10,1,7.55919251351514 +10,11,1,7.13749058686666 +10,12,1,8.4131109388189 +10,13,1,8.243239304495 +10,14,1,9.04357503034394 +11,1,0,0.690807864855916 +11,2,0,1.21305571753637 +11,3,0,1.74015621229651 +11,4,0,2.00159025849747 +11,5,0,2.11574596971279 +11,6,0,3.19092464619881 +11,7,0,3.22429735309096 +11,8,1,5.13382668549291 +11,9,1,5.9140042396338 +11,10,1,6.4637308081868 +11,11,1,6.74326537626285 +11,12,1,7.27352318583013 +11,13,1,7.25647378481152 +11,14,1,7.91144917846476 +12,1,0,-0.772883107430939 +12,2,0,0.562438790901717 +12,3,0,0.312212215701474 +12,4,0,0.915013671709732 +12,5,1,2.75432564494218 +12,6,1,3.29866181287913 +12,7,1,3.9591735430352 +12,8,1,4.71570092957779 +12,9,1,4.891367907893 +12,10,1,5.02904192826964 +12,11,1,5.5869683932088 +12,12,1,5.72413801772568 +12,13,1,6.60609431234185 +12,14,1,6.92491068601221 +13,1,0,0.667093263894111 +13,2,0,2.20240402894195 +13,3,0,1.59866462724899 +13,4,0,2.23118638730291 +13,5,0,2.76434261638602 +13,6,1,5.52180377540738 +13,7,1,5.46838523257722 +13,8,1,6.4008982696852 +13,9,1,5.78034293756765 +13,10,1,7.02728995316161 +13,11,1,7.05639155464656 +13,12,1,7.64402942507319 +13,13,1,7.66936377891581 +13,14,1,7.82217790931329 +14,1,0,-0.974894047328861 +14,2,0,-0.798908170327818 +14,3,0,-0.325615286818747 +14,4,0,0.33653392121698 +14,5,0,0.559214569226999 +14,6,0,0.828822770383645 +14,7,0,1.2768326144426 +14,8,1,3.91544620831202 +14,9,1,4.61519349884478 +14,10,1,4.17384706082031 +14,11,1,4.90013698170225 +14,12,1,5.88083622439122 +14,13,1,5.72222322923281 +14,14,1,6.55307420228637 +15,1,0,-4.21831431056225 +15,2,0,-3.63966759040906 +15,3,0,-2.92370234081679 +15,4,0,-2.51916587175183 +15,5,0,-2.94323906563132 +15,6,0,-2.11383967904404 +15,7,0,-1.50048490200698 +15,8,1,0.451732537935197 +15,9,1,1.03046170445661 +15,10,1,1.58996266093569 +15,11,1,1.52863816101507 +15,12,1,2.13685847554405 +15,13,1,2.42171685882613 +15,14,1,3.21828421240355 +16,1,0,-0.385854328048857 +16,2,0,-0.338305602132827 +16,3,0,-0.238590221798329 +16,4,0,0.743569498725612 +16,5,0,1.05203584635076 +16,6,0,1.22186089172849 +16,7,0,1.24653000589988 +16,8,1,4.44674278096485 +16,9,1,4.47510538776514 +16,10,1,4.99488312417528 +16,11,1,5.48973110039526 +16,12,1,5.62466374148908 +16,13,1,6.44512239972061 +16,14,1,6.43623480544483 +17,1,0,2.16785447628497 +17,2,0,2.31901449279009 +17,3,0,3.15671814918528 +17,4,1,4.8439928034467 +17,5,1,5.45867320185413 +17,6,1,6.28532448058888 +17,7,1,6.76022246427503 +17,8,1,7.25720108021679 +17,9,1,7.50814898259792 +17,10,1,7.9731799765222 +17,11,1,8.47774256037482 +17,12,1,8.51199928589437 +17,13,1,8.50060256662954 +17,14,1,9.08387289405168 +18,1,0,-1.60951087864456 +18,2,0,-1.3770667544635 +18,3,0,-0.483465524744515 +18,4,0,0.0783722523906124 +18,5,0,-0.361858649834909 +18,6,0,0.597870300260408 +18,7,1,2.61462661998153 +18,8,1,3.00278443230967 +18,9,1,3.39969989822973 +18,10,1,3.8353309748033 +18,11,1,4.55979334591814 +18,12,1,5.073814717008 +18,13,1,5.26220667346639 +18,14,1,5.63386231768049 +19,1,0,0.850880033120878 +19,2,0,1.15978325950994 +19,3,0,1.70852438464909 +19,4,1,4.192182824578 +19,5,1,4.42890008724135 +19,6,1,4.70373458640402 +19,7,0,3.65443718587879 +19,8,0,3.45094703589316 +19,9,0,3.48631311865056 +19,10,0,4.45134653061 +19,11,1,5.99340310135949 +19,12,1,7.5327026592693 +19,13,1,7.21922059653638 +19,14,1,7.98664833012848 +20,1,0,-0.0380996484767676 +20,2,0,0.160180858588788 +20,3,0,1.0800434902016 +20,4,0,1.2083539958016 +20,5,0,1.70134235841492 +20,6,1,3.9430473086055 +20,7,1,4.72033695526799 +20,8,0,3.12744238518949 +20,9,0,3.31822524642659 +20,10,1,6.19789711074577 +20,11,1,5.9650031200522 +20,12,1,6.35369584711167 +20,13,1,7.23990385320775 +20,14,1,6.93743260617506 +21,1,0,1.6379237130936 +21,2,0,2.07761308728722 +21,3,1,4.66201500666222 +21,4,1,5.31541230884966 +21,5,1,5.29023856209878 +21,6,0,3.79836244219868 +21,7,0,4.29516122321749 +21,8,0,4.17931288370224 +21,9,0,5.10293674890383 +21,10,1,7.4085342863041 +21,11,1,7.45915089716696 +21,12,1,7.70806198424442 +21,13,1,8.7542219462747 +21,14,1,9.09727457843362 +22,1,0,0.213810385805516 +22,2,0,-0.166803465410663 +22,3,0,0.706106049598579 +22,4,0,1.1315720067563 +22,5,1,3.40449033642509 +22,6,1,3.85162973352413 +22,7,1,4.11743659130997 +22,8,0,2.7116347836987 +22,9,0,3.09106944908372 +22,10,1,4.98955077740061 +22,11,1,5.8080208678559 +22,12,1,6.28788530323797 +22,13,1,6.36268155763008 +22,14,1,7.32199124891852 +23,1,0,1.18304195674393 +23,2,0,1.01172202506791 +23,3,0,1.46250880587674 +23,4,0,1.98074895810419 +23,5,1,3.8051069494748 +23,6,1,4.48480525581357 +23,7,1,4.43285011472359 +23,8,0,2.96779479488377 +23,9,0,4.05540289094202 +23,10,0,3.95449298906873 +23,11,0,4.61948536227856 +23,12,1,7.54810282237943 +23,13,1,6.94767528458599 +23,14,1,7.38612103736348 +24,1,0,3.94752604540854 +24,2,0,4.47282332399766 +24,3,0,5.1287489817583 +24,4,0,5.73106083327451 +24,5,1,8.34304886420349 +24,6,1,7.91639836158782 +24,7,1,8.59785535974666 +24,8,0,7.21414714762342 +24,9,0,7.82114499247611 +24,10,0,8.19485663098121 +24,11,1,10.9317794612466 +24,12,1,10.7421644700021 +24,13,1,11.4116871530707 +24,14,1,11.1048327838521 +25,1,0,1.03421003096334 +25,2,0,1.71538483679372 +25,3,0,1.64706906535895 +25,4,0,1.94031118089325 +25,5,0,2.77140308509926 +25,6,1,4.98466746356752 +25,7,1,5.2741015767921 +25,8,0,3.3457494289616 +25,9,0,3.85782555111622 +25,10,0,4.37336484449863 +25,11,1,6.75418696946998 +25,12,1,6.93752829800433 +25,13,1,8.36657135688587 +25,14,1,8.16677481357262 +26,1,0,-0.0763063158582707 +26,2,0,0.466451710918771 +26,3,0,0.910576947588877 +26,4,0,1.13322135663164 +26,5,1,4.23752640302729 +26,6,1,3.76329928551497 +26,7,0,2.51558253216842 +26,8,0,2.97425466290185 +26,9,0,2.96670991402842 +26,10,0,3.58287125991567 +26,11,1,5.82144905522926 +26,12,1,6.22983455878528 +26,13,1,6.79032481399152 +26,14,1,7.35226129995372 +27,1,0,2.27859835232144 +27,2,0,3.17476434312639 +27,3,0,3.25216303019892 +27,4,0,3.70022013068184 +27,5,1,6.27579911351737 +27,6,1,6.69289343116755 +27,7,0,4.82489131821854 +27,8,0,5.4429141080516 +27,9,0,6.08669439674896 +27,10,1,7.63422951714493 +27,11,1,8.64316242118111 +27,12,1,8.89829160590378 +27,13,1,9.25571598707512 +27,14,1,9.24569101896008 +28,1,0,-0.342450467395278 +28,2,0,0.310007754769391 +28,3,0,-0.0311344214944806 +28,4,0,0.862144985146423 +28,5,0,0.853495571395288 +28,6,1,3.65104723287197 +28,7,1,4.46118983994499 +28,8,0,1.84203063994238 +28,9,0,2.44954555935375 +28,10,0,3.00487690656486 +28,11,0,3.65054794472119 +28,12,1,5.58877278172414 +28,13,1,6.22939800867661 +28,14,1,6.79480196070151 +29,1,0,-0.276212810426299 +29,2,0,-0.19739994561799 +29,3,1,2.14162226305334 +29,4,1,3.15288165444035 +29,5,1,3.68176984908255 +29,6,0,1.46491972719422 +29,7,0,1.88701014894873 +29,8,1,4.6588643132694 +29,9,1,4.91284363064503 +29,10,1,5.45381783726839 +29,11,1,5.83377542086583 +29,12,1,6.1158399139377 +29,13,1,6.57119974013501 +29,14,1,6.93350884970635 +30,1,0,4.10016684558445 +30,2,0,4.27886637184127 +30,3,1,6.3482580199826 +30,4,1,7.12615128821499 +30,5,1,7.66322230306199 +30,6,0,5.65954403907913 +30,7,0,6.33443791161181 +30,8,0,6.51385796896648 +30,9,0,7.3735879491823 +30,10,1,9.35687106716847 +30,11,1,10.6773373935836 +30,12,1,10.2003379559713 +30,13,1,10.9311737133248 +30,14,1,11.3687107480504 +31,1,1,0.805408712047433 +31,2,1,1.84776906154693 +31,3,1,1.64778143673773 +31,4,1,2.55885102432572 +31,5,1,2.90614742580354 +31,6,1,3.00888863197211 +31,7,1,4.43049982312647 +31,8,1,3.62727297145316 +31,9,1,5.15954601334955 +31,10,1,4.91306292880111 +31,11,1,5.05324119867446 +31,12,1,6.00220702207599 +31,13,1,6.11348000405544 +31,14,1,5.96113493125594 +32,1,1,1.83946890780068 +32,2,1,1.88186863220079 +32,3,1,2.68836086024627 +32,4,1,3.23673352167703 +32,5,1,3.7085717754689 +32,6,1,3.75805860230932 +32,7,1,4.41983565908307 +32,8,1,4.11288619889539 +32,9,1,5.36975943353597 +32,10,1,5.91660584222348 +32,11,1,5.42355078771171 +32,12,1,7.00037841689709 +32,13,1,6.84124810579276 +32,14,1,7.07481809631016 +33,1,1,8.10818108152715 +33,2,1,8.03301455804785 +33,3,1,8.48622528062838 +33,4,1,9.23361830889668 +33,5,1,9.58118233583954 +33,6,1,10.0434861156558 +33,7,1,10.2712025992497 +33,8,1,10.2446228427252 +33,9,1,10.966121920323 +33,10,1,11.3306759384184 +33,11,1,11.2500184185823 +33,12,1,12.0954283704816 +33,13,1,12.4136785418305 +33,14,1,13.1515546049579 +34,1,1,1.91107230283632 +34,2,1,2.78787448056418 +34,3,1,3.30397428411788 +34,4,1,3.84930214942851 +34,5,1,3.95162059005163 +34,6,1,4.80661955757754 +34,7,1,4.5507995689622 +34,8,1,5.10052210821787 +34,9,1,5.27741970879206 +34,10,1,6.1242404637517 +34,11,1,6.14131766166448 +34,12,1,7.39763843167614 +34,13,1,6.73972732034107 +34,14,1,6.98030453854261 +35,1,1,0.814456135472172 +35,2,1,1.48106477787428 +35,3,1,1.89443441870839 +35,4,1,2.20082894362952 +35,5,1,3.10394198260418 +35,6,1,2.79428567953601 +35,7,1,3.45261099844245 +35,8,1,3.48448331359979 +35,9,1,4.00293363660516 +35,10,1,3.65495154838888 +35,11,1,4.54914887775535 +35,12,1,5.34546364663535 +35,13,1,5.62622026570335 +35,14,1,6.5577195282091 +36,1,1,2.89322338223221 +36,2,1,2.43763937366069 +36,3,1,3.52024811873703 +36,4,1,3.90112872347941 +36,5,1,4.2832049543086 +36,6,1,4.42494878019294 +36,7,1,4.95249454695203 +36,8,1,5.35786607170525 +36,9,1,5.6291677671536 +36,10,1,5.86911086605817 +36,11,1,6.43707730014866 +36,12,1,7.00418196259803 +36,13,1,7.73888465356696 +36,14,1,7.93366289599384 +37,1,1,0.385352616841546 +37,2,1,0.819603923299169 +37,3,1,1.46771231227335 +37,4,1,2.27117635114165 +37,5,1,2.11074183315829 +37,6,1,2.88801620048729 +37,7,1,3.36367893641003 +37,8,1,3.67520326994127 +37,9,1,4.05952863727388 +37,10,1,4.63061986646866 +37,11,1,5.088845080983 +37,12,1,5.34855620684275 +37,13,1,5.75545968021002 +37,14,1,5.44864859299669 +38,1,1,3.37858416892007 +38,2,1,3.15812882246318 +38,3,1,3.86386755867264 +38,4,1,4.46950057006428 +38,5,1,4.34123798580878 +38,6,1,4.65681077757064 +38,7,1,4.98926640042359 +38,8,1,5.73215972698182 +38,9,1,6.20458558873255 +38,10,1,6.93403534991197 +38,11,1,7.45987707516908 +38,12,1,7.66871997891298 +38,13,1,7.89302936562788 +38,14,1,8.38779607659706 +39,1,1,3.20095004884398 +39,2,1,3.60368009468874 +39,3,1,4.25931765946464 +39,4,1,4.7603005789472 +39,5,1,4.90872434977372 +39,6,1,5.27017351924028 +39,7,1,5.62220178377335 +39,8,1,5.48171789767303 +39,9,1,6.38885246344912 +39,10,1,6.98773198739699 +39,11,1,7.25075696732473 +39,12,1,7.84854591342661 +39,13,1,7.47896223341692 +39,14,1,8.36818293752752 +40,1,1,2.87093191154663 +40,2,1,2.87597596831069 +40,3,1,3.03180595654362 +40,4,1,3.30420197104686 +40,5,1,4.32236944871849 +40,6,1,4.97672758655547 +40,7,1,5.25245840472456 +40,8,1,4.86714431054946 +40,9,1,5.49959246949463 +40,10,1,5.52371373736801 +40,11,1,6.30165579989535 +40,12,1,6.76457636886329 +40,13,1,7.70875649312729 +40,14,1,7.11778942277854 +41,1,0,-1.11340439700276 +41,2,0,-0.638158658166492 +41,3,0,-0.778651839121851 +41,4,0,-0.150425432540447 +41,5,0,0.47365130161576 +41,6,0,0.841811450496652 +41,7,0,1.06674098962402 +41,8,0,1.82566985688518 +41,9,0,2.06798559854346 +41,10,0,2.056328320611 +41,11,0,3.30911063077778 +41,12,0,3.81971554318778 +41,13,0,3.2573993961219 +41,14,0,3.93414157225104 +42,1,0,-0.245419154315059 +42,2,0,0.165605752570345 +42,3,0,0.979445898179329 +42,4,0,0.937872487195166 +42,5,0,1.31988525577858 +42,6,0,1.8321416306157 +42,7,0,2.03645193162772 +42,8,0,2.67768557926889 +42,9,0,3.03136613617435 +42,10,0,3.6635297796973 +42,11,0,3.57958077247572 +42,12,0,4.64117645974538 +42,13,0,4.81522469701444 +42,14,0,5.05628851012022 +43,1,0,0.652818654589694 +43,2,0,1.32390229472278 +43,3,0,1.51129563983297 +43,4,0,1.78209791811164 +43,5,0,2.40160142723841 +43,6,0,2.67881560247379 +43,7,0,3.83894903275002 +43,8,0,3.992976823822 +43,9,0,4.37948449861178 +43,10,0,4.08623959752446 +43,11,0,5.12137758193446 +43,12,0,5.53269044777847 +43,13,0,5.63784509720341 +43,14,0,6.13359788425623 +44,1,0,0.00893117713676395 +44,2,0,1.34097519458324 +44,3,0,0.683214031563897 +44,4,0,1.93210026004211 +44,5,0,1.95757842728703 +44,6,0,2.39268373974504 +44,7,0,3.03417649834621 +44,8,0,3.00103235859196 +44,9,0,3.43917062475798 +44,10,0,3.98819450456049 +44,11,0,4.17133914067727 +44,12,0,4.71235573874282 +44,13,0,5.80770026144296 +44,14,0,5.79519601683128 +45,1,0,0.253457610082622 +45,2,0,1.13453153574105 +45,3,0,0.932781432285403 +45,4,0,0.975580568471785 +45,5,0,1.38584489860182 +45,6,0,2.03279763867754 +45,7,0,2.47020669915692 +45,8,0,2.95942631126133 +45,9,0,3.67737471056936 +45,10,0,3.7896415178554 +45,11,0,4.1159306954169 +45,12,0,4.41306709461098 +45,13,0,5.53508328848706 +45,14,0,5.43230375075757 +46,1,0,-1.52435376189835 +46,2,0,-1.2846591208183 +46,3,0,-0.740338515625732 +46,4,0,-0.268175038451926 +46,5,0,0.0973767669783354 +46,6,0,0.495983765562471 +46,7,0,0.602206726252663 +46,8,0,1.47879792939887 +46,9,0,1.78349218844032 +46,10,0,2.26192789602674 +46,11,0,2.31271360769465 +46,12,0,3.23335323764435 +46,13,0,2.73402133853866 +46,14,0,3.90991988487791 +47,1,0,0.876006413206764 +47,2,0,1.55616322756757 +47,3,0,0.940558713644934 +47,4,0,2.11481149159238 +47,5,0,2.36730850752517 +47,6,0,2.50379162881074 +47,7,0,2.67311162480467 +47,8,0,3.64677086170823 +47,9,0,3.78127037568889 +47,10,0,4.18271292888441 +47,11,0,4.84789556699648 +47,12,0,5.48284406726014 +47,13,0,5.42390522866612 +47,14,0,6.45252942953694 +48,1,0,-0.0456062302553433 +48,2,0,1.01951030090815 +48,3,0,1.33827613821552 +48,4,0,1.02197276148686 +48,5,0,1.7736359861221 +48,6,0,2.59167624213072 +48,7,0,3.44314955933283 +48,8,0,3.19756911916689 +48,9,0,3.55604542283138 +48,10,0,3.97932862428113 +48,11,0,4.38218359665931 +48,12,0,4.6283248455246 +48,13,0,5.16673184947322 +48,14,0,5.06219451328326 +49,1,0,0.226346955841783 +49,2,0,0.604190565046408 +49,3,0,1.70987733604714 +49,4,0,1.30731004680244 +49,5,0,1.51791411623291 +49,6,0,2.117339634813 +49,7,0,1.93660270104447 +49,8,0,2.76502404154791 +49,9,0,3.64828093303865 +49,10,0,3.60291833058595 +49,11,0,4.26609647481291 +49,12,0,4.66548405044959 +49,13,0,4.99502227583576 +49,14,0,5.09568611071855 +50,1,0,2.50727463180107 +50,2,0,3.36490283476309 +50,3,0,3.44201673808199 +50,4,0,3.61182548158305 +50,5,0,4.41913252173939 +50,6,0,5.11244520150022 +50,7,0,5.08802321104889 +50,8,0,5.61091214843808 +50,9,0,6.11469546442885 +50,10,0,6.35140111090911 +50,11,0,6.60201483310867 +50,12,0,7.37151509864027 +50,13,0,7.67524411334164 +50,14,0,8.09667833580775 +51,1,0,1.40036466309899 +51,2,0,1.59346023303238 +51,3,0,1.92355185331857 +51,4,0,2.15960411414969 +51,5,0,2.71649054681989 +51,6,0,3.05848325174544 +51,7,0,3.13419614349564 +51,8,0,3.77095513835358 +51,9,0,4.2307413131312 +51,10,0,4.17732854858233 +51,11,0,4.68030184139705 +51,12,0,5.40117740618557 +51,13,0,5.3140690569903 +51,14,0,6.04943116207046 +52,1,0,0.972185436095904 +52,2,0,1.1376376850034 +52,3,0,1.35238839751621 +52,4,0,2.49515266657083 +52,5,0,2.04010324339694 +52,6,0,2.63807905528647 +52,7,0,3.08952363566487 +52,8,0,3.81571436303112 +52,9,0,4.47881597100096 +52,10,0,4.49538687220209 +52,11,0,5.15058499423025 +52,12,0,5.07447475318914 +52,13,0,6.04209570214385 +52,14,0,6.17582801714608 +53,1,0,1.05614902097164 +53,2,0,1.09393904865078 +53,3,0,1.38277646115393 +53,4,0,2.30059744988638 +53,5,0,2.10810398882383 +53,6,0,3.38091615828707 +53,7,0,3.35508845814325 +53,8,0,4.09043702246648 +53,9,0,4.29033326184983 +53,10,0,4.1600031259291 +53,11,0,5.2197025915643 +53,12,0,5.50701180812023 +53,13,0,5.70354233182186 +53,14,0,6.27791481477089 +54,1,0,-0.249560249495564 +54,2,0,-0.638465348422011 +54,3,0,0.218390587213658 +54,4,0,0.695505113611535 +54,5,0,1.03702742088627 +54,6,0,1.62592446236301 +54,7,0,2.04414873366642 +54,8,0,2.14643869621477 +54,9,0,2.32678859570252 +54,10,0,2.89946736639923 +54,11,0,3.22372144272528 +54,12,0,3.88703555626426 +54,13,0,3.75867562262484 +54,14,0,4.61962618990118 +55,1,0,-2.55844603327382 +55,2,0,-2.4064275819275 +55,3,0,-2.14987067836445 +55,4,0,-1.41032256154091 +55,5,0,-1.44531468183849 +55,6,0,-1.01355889205471 +55,7,0,0.0232385820049242 +55,8,0,0.151401821383294 +55,9,0,0.457981875768269 +55,10,0,0.822769139903052 +55,11,0,1.51565616157214 +55,12,0,1.43128400571685 +55,13,0,2.28756410448868 +55,14,0,1.93357155804597 +56,1,0,-0.476623436752858 +56,2,0,-0.527000308457479 +56,3,0,0.43664775763061 +56,4,0,1.03247176760959 +56,5,0,0.985027370954817 +56,6,0,1.64212937180305 +56,7,0,2.00582868817396 +56,8,0,2.05607625064923 +56,9,0,2.44034740812859 +56,10,0,2.73882112658986 +56,11,0,3.31709955730561 +56,12,0,3.47520962513344 +56,13,0,4.18008591886293 +56,14,0,4.63108781888434 +57,1,0,3.42587364731153 +57,2,0,4.3689681006224 +57,3,0,4.16138790214216 +57,4,0,4.28875214096414 +57,5,0,4.95249417691202 +57,6,0,5.38720281573228 +57,7,0,5.33019363218282 +57,8,0,6.06718519860403 +57,9,0,6.67615335811295 +57,10,0,7.11950406662057 +57,11,0,7.56794263553245 +57,12,0,7.40270698993269 +57,13,0,8.03627180051018 +57,14,0,8.70273528552716 +58,1,0,2.71848491535119 +58,2,0,2.74114985126453 +58,3,0,3.82636437322039 +58,4,0,4.31363952725973 +58,5,0,4.73597673008361 +58,6,0,4.94088391066019 +58,7,0,5.4405620018625 +58,8,0,6.29700557966413 +58,9,0,6.60020275668388 +58,10,0,6.14755475477957 +58,11,0,6.85792244751181 +58,12,0,6.98453372682983 +58,13,0,7.93500383801066 +58,14,0,7.7672015099005 +59,1,0,-0.456884855752553 +59,2,0,-0.911547939660378 +59,3,0,-0.186883516436952 +59,4,0,-0.155206278546137 +59,5,0,0.70007345405345 +59,6,0,1.62412908632396 +59,7,0,1.72923533973511 +59,8,0,1.6161523563518 +59,9,0,2.26113515259204 +59,10,0,2.23604567397879 +59,11,0,3.26472384586568 +59,12,0,3.55774182983386 +59,13,0,3.66144836634766 +59,14,0,3.68534769097932 +60,1,0,-1.64803753572784 +60,2,0,-1.19621887542572 +60,3,0,-0.6188191487774 +60,4,0,0.24646642209756 +60,5,0,0.671004352563786 +60,6,0,0.834332656953114 +60,7,0,0.528487229115304 +60,8,0,1.55793836010352 +60,9,0,2.08343982850051 +60,10,0,2.29429220979833 +60,11,0,2.46181447933827 +60,12,0,3.15161125776922 +60,13,0,3.45669913943692 +60,14,0,3.48176844595534 diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 97245e5a..ee7e9db3 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1859,7 +1859,7 @@ The paper specifies no standard-error formula (Section 1 defers to "standard, we 1. **Note:** Standard errors are **cluster-robust at the unit level by default** - `cluster=None` auto-clusters at the unit identifier and the results record `cluster_name`/`n_clusters` - with a `t(G-1)` reference distribution (G = realized clusters in each horizon's clean-control sample). Matches Stata `lpdid` `vce(cluster unit)`; the paper prescribes no SE. 2. **Note:** The regression-adjustment (RA) covariate path (`reweight=True` with covariates/absorb) reports an **influence-function cluster variance** `sum_c (sum_{i in c} psi_i)^2 / n^2`, in the same family as `ImputationDiD`'s Theorem-3 / BJS variance (see "IF-based variance estimators vs analytical-sandwich estimators" above). Its single Gram inversion is routed through `linalg._rank_guarded_inv` (finite SE on the identified subspace under near-collinearity; NaN at rank 0). Unlike the default/weighted `solve_ols` `hc1`-cluster path - which applies the `(G/(G-1))*((n-1)/(n-k))` finite-sample factor - the RA IF variance carries **no finite-sample factor**, while both paths share the `t(G-1)` reference. **PR-B2 validated this asymmetry as faithful to the authors' own tooling**, not a defect: the no-factor RA convention matches the canonical Stata `teffects ra ... atet vce(cluster)` (the authors' `lpdid_regression_adjustment.do` `margins`/`kmatch` degrees-of-freedom comments prove `teffects` applies neither factor), while the default path matches `feols`/`reghdfe`. The RA *point* estimate is R-anchored to ~1e-13 (full-interaction `i.dtreat##(i.time c.x)` == `teffects` point; `tests/test_methodology_lpdid.py::test_ra_covariate_point`). The RA *standard error* itself has **no runnable R reference** (no R package computes the RA IF variance - `alexCardazzi` uses direct covariate inclusion, not RA; the canonical RA SE is Stata `teffects` only), so it is **pinned** as a documented regression value (`test_ra_covariate_se_regression_pin`) and its calibration is validated by the ungated Monte-Carlo coverage study `benchmarks/python/coverage_lpdid_ra.py` (~0.95 empirical coverage of the true effect at cluster counts G in {30, 100, 300}). 3. **Note:** Direct covariate inclusion (`reweight=False` with covariates/absorb) emits a `UserWarning`: per online Appendix B.2.2 it preserves the non-negative LP-DiD weighting result only under linear and homogeneous covariate effects, so the regression-adjustment path (`reweight=True`) is preferred. -4. **Deviation from R:** Scope - non-absorbing treatment (Section 4.2) is implemented for the **entry-effect** estimands (`non_absorbing="first_entry"` / `"effect_stabilization"`, PR-C1); the Appendix-C exit-event dynamics and survey-design support remain deferred follow-ups. Validation against the reference R packages (`alexCardazzi/lpdid` `nonabsorbing` / `nonabsorbing_lag`) is the planned PR-C2; until then the non-absorbing standard errors are the same cluster-robust family as the absorbing path (an implementation choice, validated against R in PR-C2). +4. **Deviation from R:** Scope - non-absorbing treatment (Section 4.2) implements the **entry-effect** estimands (`non_absorbing="first_entry"` / `"effect_stabilization"`, PR-C1). **PR-C2 R-parity-validated both modes against an INDEPENDENT `fixest::feols` reconstruction of the paper's Eq. 12 / Eq. 13 clean-sample restrictions** (point and SE match to ~1e-13/~1e-15 for the variance-weighted variants; the `effect_stabilization` reweighted point matches and its SE is pinned as a regression guard - a small weighted-cluster convention difference vs feols; `tests/test_methodology_lpdid.py::TestLPDiDNonAbsorbingParityR`). The recipe's independence was demonstrated when an earlier draft's Eq. 12 control off-by-one diverged from the already-correct library and was corrected against the paper, plus a hand-computed Python micro-check. **`alexCardazzi/lpdid`'s `nonabsorbing_lag` is NOT a faithful Eq. 13** (it clamps `treat_diff[<0]<-0`, so its clean-control window blocks only treatment turn-*ons*; it reuses a forward placebo window; and it NA-excludes pre-panel-treated rows where the library clamps pre-`min_t` to untreated): it diverges ~0.01-0.05 from Eq. 13 even on a monotone no-off-switch panel, so it is **recorded in the golden `meta` as a divergent third-party reference, not a parity gate** (the alexCardazzi-pooled precedent). The library's "no treatment change" (both directions) and backward placebo window are the more paper-faithful choices. `first_entry` (Eq. 12) has no R-package analogue (anchored on the independent feols recipe only). Appendix-C exit-event dynamics, the Stata canonical SE, and survey-design support remain deferred follow-ups. 5. **Note:** LP-DiD's per-unit quantities (outcome lags `ylags`, first-difference lags `dylags`, integer-`pmd` premean baselines, treatment-entry detection) are **calendar** quantities (`t-1`, `t-k`), so the estimator requires integer-valued, globally consecutive `time` labels. A unit with an **interior time gap** is handled by reindexing that unit to its complete interior calendar grid `[min_t, max_t]`, computing the features on the grid, then **restricting back to the observed rows** - so a lag/first-difference spanning a gap is NaN and the observation fails closed (never the previous-*observed* row), and no synthetic gap row enters a regression. A gap-free panel skips this entirely and is bit-identical. **Entry = first OBSERVED treated period** (`min(t | D_it=1)`): an unobserved pre-onset gap cannot move a cohort earlier, the only well-defined convention when the true switch falls in an unobserved period. 6. **Note (pooled estimand):** The pooled pre/post ATT (the headline `results.att` is the pooled-post row) is the **unit-equal-weighted average of each unit-event-time's mean long difference** over the window - `mean_h(y_{i,t+h}) - baseline_{i,t}`, one observation per (unit, event-time), regressed on the treatment-switch indicator with event-time fixed effects on the **fixed-composition** sample (only units observing *every* pooled target, with clean controls required through `max(h)`). This equals the mean of the per-horizon event-study coefficients on a balanced panel. **PR-B2 validated it against the authors' runnable R reference**: the pooled estimand matches the authors' own R pooled recipe (`danielegirardi/lpdid`: a `slider` window-mean minus `y_{t-1}` on the clean-through-window-end sample) to ~1e-13 (`tests/test_methodology_lpdid.py::test_pooled`). A prior version of this note speculated the authors used a horizon-**stacked** pooled regression; the authors' R reference in fact uses this same fixed-composition mean-long-difference, so that speculation was incorrect. Unlike the event-study variants (where `alexCardazzi` is a cross-check gate), pooled is anchored to the authors' recipe **only**: `alexCardazzi`'s pooled uses a **laxer** clean-control window, so it differs and is recorded in the golden `meta` for transparency, not as a parity target. 7. **Deviation from R:** `no_composition` is intentionally more faithful to the paper's fixed-composition intent (Section 3.6) than the R packages: it fixes the realized sample across *all* post horizons (every post coefficient shares one sample, even on unbalanced panels) and excludes cohorts with `p_g > T-H`, whereas `alexCardazzi/lpdid` uses a looser per-horizon sample and a stricter `treat_date < T-H` cutoff. It therefore has **no exact R-package anchor** and is validated by the pure-Python tests in `tests/test_lpdid.py` (the R-parity golden omits it; `alexCardazzi`'s looser-semantics value is recorded in the golden `meta`). @@ -1878,7 +1878,8 @@ The paper specifies no standard-error formula (Section 1 defers to "standard, we - [x] B1 pure-Python tests: analytical DGPs + cross-estimator equivalence (CS / BJS / DiD; Cengiz-stacked dropped, documented) + unbalanced / interior-gap / RA-overlap / pmd-missing edge cases (PR-B1) - [x] B2: self-generated R-parity (authors' `danielegirardi/lpdid` recipes + `alexCardazzi/lpdid` cross-check; VW / reweight / pmd / direct / pooled / RA-point to ~1e-12; RA SE pinned + MC-coverage-validated; `no_composition` more paper-faithful than R, B1-tested) (PR-B2) - [x] Non-absorbing extension (Section 4.2): entry-effect estimands - first-entry (Eq. 12) + effect-stabilization (Eq. 13, window `L`) via `non_absorbing`; mode-aware clean-sample masks, `C=0`-below-`min_t` boundary convention, gap-free requirement; pure-Python tests (absorbing reduction, re-entry mechanism, placebo, no-negative-weighting, stabilized-control, DGP recovery) (PR-C1) -- [ ] Non-absorbing R-parity vs `alexCardazzi/lpdid` + exit-event dynamics (Appendix C `eta_h`) - deferred (PR-C2) +- [x] Non-absorbing R-parity: both modes vs an independent `fixest::feols` Eq. 12/13 reconstruction (point+SE ~1e-13/~1e-15 vw; reweighted point + pinned SE); `alexCardazzi nonabsorbing_lag` recorded as a divergent reference (not a gate); absorbing B2 goldens byte-identical (PR-C2) +- [ ] Non-absorbing exit-event dynamics (Appendix C `eta_h`) + the Stata canonical RA/SE - deferred - [ ] Survey-design support - deferred to a later PR --- diff --git a/tests/test_methodology_lpdid.py b/tests/test_methodology_lpdid.py index ac3b3bb3..07b4b63c 100644 --- a/tests/test_methodology_lpdid.py +++ b/tests/test_methodology_lpdid.py @@ -45,6 +45,16 @@ PANEL_PATH = _DATA / "lpdid_test_panel.csv" _R_FIXTURE_AVAILABLE = GOLDEN_PATH.is_file() and PANEL_PATH.is_file() +# Phase C2: non-absorbing R-parity (a separate panel + golden so the absorbing +# fixtures above stay byte-identical). Both non-absorbing modes are anchored on an +# INDEPENDENT fixest::feols reconstruction of the paper's Eq. 12 (first_entry) and +# Eq. 13 (effect_stabilization) clean-sample restrictions; alexCardazzi's +# nonabsorbing_lag is recorded in the golden meta as a divergent reference, NOT a +# parity gate (it clamps off-switches + uses a non-paper boundary/window convention). +NA_GOLDEN_PATH = _DATA / "lpdid_nonabsorbing_golden.json" +NA_PANEL_PATH = _DATA / "lpdid_nonabsorbing_panel.csv" +_NONABSORB_FIXTURE_AVAILABLE = NA_GOLDEN_PATH.is_file() and NA_PANEL_PATH.is_file() + # Library regression-adjustment influence-function SE per horizon (reweight=True, # covariates=["x"]). NOT R-parity: the canonical RA SE is Stata teffects only, with # no runnable R analogue (see module docstring / REGISTRY ## LPDiD Deviation 2). @@ -64,6 +74,24 @@ # Headline (pooled) RA influence-function SE, same convention as RA_SE_PIN. POOLED_RA_SE_PIN = {"post": 0.35917407532121975, "pre": 0.3785290141761256} +# Non-absorbing reweighted (effect_stabilization) SE per horizon. NOT feols-parity: +# the reweighted POINT matches the independent feols recipe to ~1e-13, but the +# weighted-cluster SE has a small feols convention difference (~5e-5), so the library +# value is pinned as a regression guard (the variance-weighted SEs ARE feols-parity). +# Refresh from the library if the committed non-absorbing panel changes: fit +# LPDiD(pre_window=3, post_window=4, reweight=True, cluster="unit", +# non_absorbing="effect_stabilization", stabilization_window=3) on +# lpdid_nonabsorbing_panel.csv and read event_study["se"]. +RW_SE_PIN = { + -3: 0.0645474561163424, + -2: 0.04782093937871335, + 0: 0.07250480396311137, + 1: 0.06781253723720812, + 2: 0.08001875551196519, + 3: 0.07730466850470008, + 4: 0.09296215631711323, +} + @pytest.fixture(scope="module") def golden() -> dict: @@ -84,6 +112,24 @@ def panel() -> pd.DataFrame: return pd.read_csv(PANEL_PATH) +@pytest.fixture(scope="module") +def na_golden() -> dict: + if not _NONABSORB_FIXTURE_AVAILABLE: + pytest.skip( + "Non-absorbing R parity fixture not present. Run " + "`Rscript benchmarks/R/generate_lpdid_golden.R`." + ) + with NA_GOLDEN_PATH.open("r") as f: + return json.loads(f.read()) + + +@pytest.fixture(scope="module") +def na_panel() -> pd.DataFrame: + if not _NONABSORB_FIXTURE_AVAILABLE: + pytest.skip("Non-absorbing R parity fixture not present.") + return pd.read_csv(NA_PANEL_PATH) + + def _es_map(res) -> dict: return { int(r.horizon): (float(r.coefficient), float(r.se)) for r in res.event_study.itertuples() @@ -226,3 +272,80 @@ def test_ra_covariate_se_regression_pin(self, panel: pd.DataFrame) -> None: ), "pooled-post RA SE drift" pre_se = float(res.pooled.loc[res.pooled["window"] == "pre", "se"].iloc[0]) assert pre_se == pytest.approx(POOLED_RA_SE_PIN["pre"], abs=1e-6), "pooled-pre RA SE drift" + + +class TestLPDiDNonAbsorbingParityR: + """Pin the C1 non-absorbing modes against the independent feols Eq. 12/13 recipes. + + The variance-weighted variants match the feols golden on point AND SE; the + reweighted point matches feols while its SE is a pinned regression guard + (documented weighted-cluster convention difference). alexCardazzi's + ``nonabsorbing_lag`` is recorded in the golden meta as a divergent reference only. + """ + + def _meta(self, na_golden: dict): + m = na_golden["meta"] + return int(m["L"]), int(m["pre_window"]), int(m["post_window"]) + + def test_first_entry_event_study(self, na_golden: dict, na_panel: pd.DataFrame) -> None: + L, pre, post = self._meta(na_golden) + res = LPDiD( + pre_window=pre, post_window=post, cluster="unit", non_absorbing="first_entry" + ).fit(na_panel, outcome="y", unit="unit", time="time", treatment="treat", only_event=True) + _assert_es(_es_map(res), na_golden["first_entry"]) # point + SE (feols-parity) + + def test_effect_stabilization_event_study( + self, na_golden: dict, na_panel: pd.DataFrame + ) -> None: + L, pre, post = self._meta(na_golden) + res = LPDiD( + pre_window=pre, + post_window=post, + cluster="unit", + non_absorbing="effect_stabilization", + stabilization_window=L, + ).fit(na_panel, outcome="y", unit="unit", time="time", treatment="treat", only_event=True) + _assert_es(_es_map(res), na_golden["effect_stab"]) # point + SE (feols-parity) + + def test_effect_stabilization_reweighted_point( + self, na_golden: dict, na_panel: pd.DataFrame + ) -> None: + """Reweighted (equally-weighted) effect_stabilization POINT == feols recipe. + + The reweighted SE is pinned separately (RW_SE_PIN) - it has a small + weighted-cluster convention difference vs feols, not feols-parity. + """ + L, pre, post = self._meta(na_golden) + res = LPDiD( + pre_window=pre, + post_window=post, + reweight=True, + cluster="unit", + non_absorbing="effect_stabilization", + stabilization_window=L, + ).fit(na_panel, outcome="y", unit="unit", time="time", treatment="treat", only_event=True) + _assert_es(_es_map(res), na_golden["effect_stab_rw"], check_se=False) + # Reweighted SE: pinned regression guard (documented feols convention diff). + es = {int(r.horizon): float(r.se) for r in res.event_study.itertuples()} + for h, pinned in RW_SE_PIN.items(): + assert h in es, f"missing horizon {h}" + assert np.isfinite(es[h]) and es[h] > 0, f"h={h}: reweighted SE not finite/positive" + assert es[h] == pytest.approx(pinned, abs=1e-6), f"reweighted SE pin drift at h={h}" + + def test_alex_recorded_as_divergent_reference(self, na_golden: dict) -> None: + """alexCardazzi nonabsorbing_lag is recorded (meta), NOT a parity gate: it + diverges from the paper-faithful Eq.13 even on this panel.""" + alex = na_golden["alex_nonabsorbing_lag"] + assert "error" not in alex, f"alex reference failed to generate: {alex}" + es = na_golden["effect_stab"] + # At least one post horizon differs materially (off-switch clamp / boundary), + # which is exactly why alex is documented, not gated. + diffs = [ + abs(_num(alex[k][0]) - _num(es[k][0])) + for k in es + if int(k) >= 0 + and k in alex + and np.isfinite(_num(es[k][0])) + and np.isfinite(_num(alex[k][0])) + ] + assert diffs and max(diffs) > 1e-3, "expected a documented alex divergence on post horizons" From b29cc455afd0df2dc23abeea1010ec19cc4765a3 Mon Sep 17 00:00:00 2001 From: igerber Date: Mon, 29 Jun 2026 17:56:18 -0400 Subject: [PATCH 2/3] docs(lpdid): address C2 review P3s - pin monotone divergence, fix stale notes - Pin the "alexCardazzi diverges even without off-switches" claim with committed evidence: the generator now records `meta.alex_monotone_post_divergence` (max |alex - Eq.13| over post horizons on the monotone no-off-switch sub-panel, 0.0344), and the test asserts it > 1e-3. The divergence is not only off-switch handling. - Generator header lists the two new non-absorbing output files. - Drop the now-stale "R-package parity (PR-C2) is a follow-up" wording from the C1 CHANGELOG entry and docs/api/lpdid.rst (this work adds it); exit-event dynamics, the Stata canonical SE, and survey support remain the deferred follow-ups. Absorbing B2 goldens remain byte-identical after regeneration. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 3 +- benchmarks/R/generate_lpdid_golden.R | 42 ++++++++++++++----- .../data/lpdid_nonabsorbing_golden.json | 3 +- docs/api/lpdid.rst | 9 ++-- tests/test_methodology_lpdid.py | 4 ++ 5 files changed, 45 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 31e14293..c567ac9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,7 +27,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 non-absorbing input, bit-for-bit identical results). Both modes implement the entry-effect estimands with mode-aware clean-sample masks, a documented "untreated before the first observed period" boundary convention, and a gap-free-panel requirement; the Appendix-C - exit-event dynamics, R-package parity (PR-C2), and survey-design support remain follow-ups. + exit-event dynamics and survey-design support remain follow-ups (R-parity is covered by + the entry above). Pure-Python validation covers the absorbing reduction, the re-entry mechanism, pre-trend placebos, non-negative weighting, stabilized-control admission, and DGP recovery. - **`LPDiD` R-parity validation (absorbing).** `tests/test_methodology_lpdid.py` pins the diff --git a/benchmarks/R/generate_lpdid_golden.R b/benchmarks/R/generate_lpdid_golden.R index d95b9c27..b8d71c3a 100644 --- a/benchmarks/R/generate_lpdid_golden.R +++ b/benchmarks/R/generate_lpdid_golden.R @@ -22,8 +22,10 @@ # coverage study benchmarks/python/coverage_lpdid_ra.py. See REGISTRY.md "## LPDiD". # # Outputs (checked into the repo): -# benchmarks/data/lpdid_test_panel.csv -# benchmarks/data/lpdid_golden.json +# benchmarks/data/lpdid_test_panel.csv (absorbing panel) +# benchmarks/data/lpdid_golden.json (absorbing goldens) +# benchmarks/data/lpdid_nonabsorbing_panel.csv (non-absorbing panel, Phase C2) +# benchmarks/data/lpdid_nonabsorbing_golden.json (non-absorbing goldens, Phase C2) # # Usage: # Rscript benchmarks/R/generate_lpdid_golden.R @@ -396,13 +398,13 @@ na_clean_sample <- function(dt, mode, h) { .(time = time, role = role) }, by = unit][!is.na(role)] } -na_es_one <- function(h, reweight = FALSE) { - cs <- na_clean_sample(na_dt, na_mode, h) +na_es_one <- function(h, reweight = FALSE, dt = na_dt) { + cs <- na_clean_sample(dt, na_mode, h) if (nrow(cs) == 0L) return(c(NA_real_, NA_real_)) - d <- merge(na_dt, cs, by = c("unit", "time")) + d <- merge(dt, cs, by = c("unit", "time")) d[, `:=`(tb = time - 1L, tt = time + h)] - d <- merge(d, na_dt[, .(unit, tb = time, yb = y)], by = c("unit", "tb")) - d <- merge(d, na_dt[, .(unit, tt = time, yt = y)], by = c("unit", "tt")) + d <- merge(d, dt[, .(unit, tb = time, yb = y)], by = c("unit", "tb")) + d <- merge(d, dt[, .(unit, tt = time, yt = y)], by = c("unit", "tt")) d[, Dy := yt - yb]; d[, tr := as.integer(role == "T")] d[, has_c := any(tr == 0L), by = time]; d <- d[has_c == TRUE] # drop event-times with no clean control if (uniqueN(d$tr) < 2L) return(c(NA_real_, NA_real_)) @@ -415,11 +417,11 @@ na_es_one <- function(h, reweight = FALSE) { } c(unname(coef(m)["tr"]), unname(se(m)["tr"])) } -na_es <- function(mode, reweight = FALSE) { +na_es <- function(mode, reweight = FALSE, dt = na_dt) { na_mode <<- mode out <- list() - for (h in 0:NA_POST) out[[as.character(h)]] <- na_es_one(h, reweight) - for (h in 2:NA_PRE) out[[as.character(-h)]] <- na_es_one(-h, reweight) + for (h in 0:NA_POST) out[[as.character(h)]] <- na_es_one(h, reweight, dt) + for (h in 2:NA_PRE) out[[as.character(-h)]] <- na_es_one(-h, reweight, dt) out } @@ -427,6 +429,23 @@ first_entry_es <- na_es("eq12") effect_stab_es <- na_es("eq13") effect_stab_rw_es <- na_es("eq13", reweight = TRUE) +# ---- monotone (no-off-switch) slice: PIN the "alex diverges even without off-switches" +# claim with committed evidence. On units whose treatment never decreases, alex's +# off-switch clamp is inert, yet alex still diverges from the paper-faithful Eq.13 (its +# non-paper boundary/window convention), so the recorded max post-horizon |alex - Eq.13| +# is well above 0 -> documents that the divergence is NOT only off-switch handling. +na_mono <- na_dt[, if (all(diff(treat) >= 0L)) .SD, by = unit] # drop units with any turn-off +na_mode <<- "eq13" +mono_ours <- vapply(0:NA_POST, function(h) na_es_one(h, FALSE, na_mono)[1], numeric(1)) +mono_alex <- tryCatch({ + am <- lpdid::lpdid(as.data.frame(na_mono), window = c(-NA_PRE, NA_POST), y = "y", + unit = "unit", time = "time", treat_status = "treat", + cluster = "unit", nonabsorbing_lag = NA_L) + am$coeftable[(NA_PRE + 1L):(NA_PRE + 1L + NA_POST), "Estimate"] # rows for h = 0..POST +}, error = function(e) rep(NA_real_, NA_POST + 1L)) +alex_monotone_divergence <- max(abs(mono_ours - mono_alex), na.rm = TRUE) +message(sprintf("alex monotone-slice max|alex - Eq.13| (post h) = %.4f", alex_monotone_divergence)) + # ---- alexCardazzi nonabsorbing_lag: divergent reference (recorded, NOT gated) ---- na_alex <- tryCatch({ a <- lpdid::lpdid(as.data.frame(na_dt), window = c(-NA_PRE, NA_POST), y = "y", @@ -473,7 +492,8 @@ na_golden <- list( "Boundary convention (closes the REGISTRY 'confirm vs R in PR-C2' item): periods", "before a unit's first observed period are treated as untreated/no-change (the", "library clamps pre-min_t to 0); alexCardazzi NA-excludes such rows (first-row lag", - "is NA). A documented divergence, not a defect.") + "is NA). A documented divergence, not a defect."), + alex_monotone_post_divergence = alex_monotone_divergence # max|alex - Eq.13| over post h on the monotone (no-off-switch) sub-panel; > 0 shows the divergence is not only off-switch handling ), first_entry = first_entry_es, effect_stab = effect_stab_es, diff --git a/benchmarks/data/lpdid_nonabsorbing_golden.json b/benchmarks/data/lpdid_nonabsorbing_golden.json index 02fa9d22..26e355c9 100644 --- a/benchmarks/data/lpdid_nonabsorbing_golden.json +++ b/benchmarks/data/lpdid_nonabsorbing_golden.json @@ -13,7 +13,8 @@ "reweight_se_note": "effect_stab_rw: POINT matches the library to ~1e-13; the reweighted SE has a small feols-weighted-cluster convention difference (~5e-5) vs the library, so the library reweighted SE is pinned as a regression guard on the Python side, not asserted against feols.", "first_entry_note": "first_entry (Eq. 12) has NO R-package analogue (alexCardazzi only exposes the effect-stabilization nonabsorbing_lag). It is anchored on the independent feols Eq. 12 recipe + a hand-computed Python micro-check. Its absorbing reduction (first_entry == absorbing on absorbing panels) is a separate internal check, NOT an R anchor for non-absorbing behavior.", "alex_note": "alexCardazzi::lpdid() `nonabsorbing_lag` is NOT a faithful paper Eq. 13: it clamps treat_diff[<0]<-0 (its clean-control window blocks only treatment turn-ONS, so a unit turning OFF inside [t-L,t+h] still counts as a control) and reuses a forward window for placebos. The library uses the literal 'no treatment change' (both directions) + the backward placebo window [t-max(L,-h),t-1] (more paper-faithful). alex diverges ~0.01-0.05 from Eq. 13 even on a monotone no-off-switch panel, so it is recorded below for transparency only, NOT used as a parity gate.", - "boundary_note": "Boundary convention (closes the REGISTRY 'confirm vs R in PR-C2' item): periods before a unit's first observed period are treated as untreated/no-change (the library clamps pre-min_t to 0); alexCardazzi NA-excludes such rows (first-row lag is NA). A documented divergence, not a defect." + "boundary_note": "Boundary convention (closes the REGISTRY 'confirm vs R in PR-C2' item): periods before a unit's first observed period are treated as untreated/no-change (the library clamps pre-min_t to 0); alexCardazzi NA-excludes such rows (first-row lag is NA). A documented divergence, not a defect.", + "alex_monotone_post_divergence": 0.03442711738728 }, "first_entry": { "0": [1.846815337457, 0.09153686444445], diff --git a/docs/api/lpdid.rst b/docs/api/lpdid.rst index 49e99f80..ad503abd 100644 --- a/docs/api/lpdid.rst +++ b/docs/api/lpdid.rst @@ -24,9 +24,12 @@ estimand is a strictly non-negatively-weighted average of cohort effects. ``stabilization_window=L`` and lets units whose treatment has been stable for at least ``L`` periods serve as clean controls — feasible with few or no never-treated units). Non-absorbing modes require a gap-free panel within - each unit's observed span and cover the entry-effect estimands; the - Appendix-C exit-event dynamics, R-package parity, and survey-design support - are planned follow-ups. Covariates and absorbed fixed + each unit's observed span and cover the entry-effect estimands. The + non-absorbing entry-effect paths are R-parity-validated against an independent + ``fixest::feols`` reconstruction of the paper's Eq. 12/13 (see + ``docs/methodology/REGISTRY.md``); the Appendix-C exit-event dynamics, the + Stata canonical SE, and survey-design support remain planned follow-ups. + Covariates and absorbed fixed effects are supported; under ``reweight=False`` they enter by direct inclusion, which preserves the non-negative weighting result only under homogeneous covariate effects (online Appendix B.2.2) — the diff --git a/tests/test_methodology_lpdid.py b/tests/test_methodology_lpdid.py index 07b4b63c..6f200819 100644 --- a/tests/test_methodology_lpdid.py +++ b/tests/test_methodology_lpdid.py @@ -349,3 +349,7 @@ def test_alex_recorded_as_divergent_reference(self, na_golden: dict) -> None: and np.isfinite(_num(alex[k][0])) ] assert diffs and max(diffs) > 1e-3, "expected a documented alex divergence on post horizons" + # The divergence is NOT only off-switch handling: it persists on the monotone + # (no-off-switch) sub-panel, recorded as committed evidence in meta. + mono = _num(na_golden["meta"]["alex_monotone_post_divergence"]) + assert mono > 1e-3, f"expected alex to diverge from Eq.13 on the monotone slice, got {mono}" From e8eb117dfab8c2eecd765c6defd410c933f902b6 Mon Sep 17 00:00:00 2001 From: igerber Date: Mon, 29 Jun 2026 17:59:50 -0400 Subject: [PATCH 3/3] docs(lpdid): de-stale the non-absorbing boundary-convention note (C2 review P3) REGISTRY's non-absorbing edge-case bullet said the pre-panel boundary convention was "an assumption to confirm against R in PR-C2"; PR-C2 has now documented it as a known divergence from alexCardazzi (which NA-excludes such first-rows). Point to Deviation #4 instead of the now-complete forward reference. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/methodology/REGISTRY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index ee7e9db3..37b665f9 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1850,7 +1850,7 @@ Eq. 12 reuses the absorbing clean control and only restricts the treated set (a - **Bias-variance (Sections 3.3, 5.3):** variance weighting (default) -> lower variance, some bias; equal weighting (`reweight`) -> unbiased, higher variance. Variance won at short horizons, equal at long horizons in the paper's simulation. - **PMD vs first-lag (Section 3.4):** PMD gains efficiency under low autocorrelation but can amplify bias if PT holds only in some pretreatment periods; first-lag relies on weaker PT (Marcus & Sant'Anna 2021). Choose the base period ex-ante. - **Covariate-weight positivity (online Appendix B.2):** direct covariate inclusion keeps non-negative weights ONLY under linear + homogeneous covariate effects (B.2.1; main-text Assumption 6); in the general case (B.2.2) weights are not guaranteed positive -> prefer the RA covariate path (the direct path should carry a homogeneity-assumption warning). -- **Non-absorbing (Section 4.2, online Appendix C):** implemented via `non_absorbing="first_entry"` (Eq. 12) and `non_absorbing="effect_stabilization"` (Eq. 13, requires `stabilization_window=L`); the default `non_absorbing=None` keeps the absorbing path and still rejects non-absorbing input. Both modes are **entry-effect** estimands; the Appendix-C exit-event dynamics (`eta_h^{g,n}`, separate switch-off event-studies) are a deferred follow-up. **Boundary convention:** periods before a unit's first observed period are treated as untreated with no change (extends Deviation 5), so window conditions clamp pre-`min_t` offsets to 0 - a unit genuinely treated before the panel starts could be misread as a fresh entry under `effect_stabilization` (an assumption to confirm against R in PR-C2). **Interior gaps** make the `[t-L, t+h]` window conditions unverifiable, so non-absorbing modes require gap-free panels within each unit's observed span and raise otherwise (the absorbing path's interior-gap reindex is a deferred follow-up for non-absorbing). +- **Non-absorbing (Section 4.2, online Appendix C):** implemented via `non_absorbing="first_entry"` (Eq. 12) and `non_absorbing="effect_stabilization"` (Eq. 13, requires `stabilization_window=L`); the default `non_absorbing=None` keeps the absorbing path and still rejects non-absorbing input. Both modes are **entry-effect** estimands; the Appendix-C exit-event dynamics (`eta_h^{g,n}`, separate switch-off event-studies) are a deferred follow-up. **Boundary convention:** periods before a unit's first observed period are treated as untreated with no change (extends Deviation 5), so window conditions clamp pre-`min_t` offsets to 0 - a unit genuinely treated before the panel starts could be misread as a fresh entry under `effect_stabilization` (PR-C2 documented this as a known divergence from `alexCardazzi/lpdid`, which NA-excludes such first-rows - see Deviation #4). **Interior gaps** make the `[t-L, t+h]` window conditions unverifiable, so non-absorbing modes require gap-free panels within each unit's observed span and raise otherwise (the absorbing path's interior-gap reindex is a deferred follow-up for non-absorbing). ### Deviations from the paper / from R / library extensions