Skip to content

Add LPDiD R-parity validation harness (Dube et al. 2025), Phase B2#583

Merged
igerber merged 1 commit into
mainfrom
feature/lpdid-r-parity
Jun 29, 2026
Merged

Add LPDiD R-parity validation harness (Dube et al. 2025), Phase B2#583
igerber merged 1 commit into
mainfrom
feature/lpdid-r-parity

Conversation

@igerber

@igerber igerber commented Jun 29, 2026

Copy link
Copy Markdown
Owner

Summary

Phase B2 of the LPDiD initiative: a self-generated R-parity validation harness for the
absorbing LPDiD estimator (shipped in 3.6.0). No estimator code changes — a
verify-and-document PR.

  • benchmarks/R/generate_lpdid_golden.R — in-R panel (staggered absorbing, ~63% never-treated,
    heterogeneous, + an interior-gap unit) and 6 golden variants computed from the method authors'
    own R recipes (danielegirardi/lpdid event-study / reweight / premean / pooled fixest::feols
    specifications), with an alexCardazzi/lpdid cross-check gate (pinned commit ba64563,
    fail-closed). Writes committed lpdid_test_panel.csv + lpdid_golden.json.
  • tests/test_methodology_lpdid.py — skip-guarded parity (matching the didimputation precedent):
    variance-weighted / reweight / pmd / direct-covariate / pooled / RA-point match to ~1e-12
    (asserted at 1e-6/1e-7 for cross-platform robustness, green on both Python and Rust backends);
    RA influence-function SE pinned (event-study + headline pooled).
  • benchmarks/python/coverage_lpdid_ra.py + lpdid_ra_coverage.json — ungated Monte-Carlo
    coverage study (mirrors coverage_sdid.py) validating the RA SE: ≈0.95 empirical coverage of
    both the event-study horizons and the headline pooled-row RA CI across cluster counts
    G ∈ {30, 100, 300}.
  • Resolves the two previously-provisional REGISTRY deviation notes (Add fixed effects and absorb parameters to DifferenceInDifferences #2 RA SE convention, Add multi-period DiD support #6 pooled
    fixed-composition estimand) in the library's favour; documents no_composition as more
    paper-faithful than the R packages (Prepare v0.2.0 release #7); ticks the REGISTRY B2 checklist box.

Methodology references

  • Method name(s): LPDiD (Local Projections Difference-in-Differences)
  • Paper / source link(s): Dube, Girardi, Jordà & Taylor (2025), J. Applied Econometrics
    40(5):741-758, https://doi.org/10.1002/jae.70000. Reference software: the authors'
    danielegirardi/lpdid (R + Stata) and the third-party alexCardazzi/lpdid (pinned ba64563).
  • Intentional deviations (documented in docs/methodology/REGISTRY.md ## LPDiD):
    • RA standard error (Add fixed effects and absorb parameters to DifferenceInDifferences #2): the regression-adjustment influence-function cluster variance
      carries no finite-sample factor — matching the authors' canonical Stata
      teffects ra ... atet vce(cluster) convention (their .do dof comments). No R package
      computes this SE, so the RA point is R-anchored (~1e-12) and the SE is pinned +
      Monte-Carlo-coverage-validated; a tracked TODO.md follow-up is to contribute a runnable R RA
      reference (only trusted once cross-checked vs Stata teffects).
    • Pooled estimand (Add multi-period DiD support #6): fixed-composition mean-long-difference — validated to match the
      authors' R pooled recipe to ~1e-13 (corrects the prior "horizon-stacked" wording).
      alexCardazzi's pooled uses a laxer clean-control window (recorded in meta, not gated).
    • no_composition (Prepare v0.2.0 release #7): more faithful to the paper's fixed-composition intent than the R
      packages (fixes the realized sample across all post horizons) → no exact R anchor; validated
      by the pure-Python B1 tests in tests/test_lpdid.py.

Validation

  • Tests added/updated: tests/test_methodology_lpdid.py (8 tests; skip-guarded R-parity, green on
    Python + Rust backends). Regenerate goldens via Rscript benchmarks/R/generate_lpdid_golden.R.
  • Simulation evidence: benchmarks/python/coverage_lpdid_ra.py
    benchmarks/data/lpdid_ra_coverage.json (RA SE empirical coverage ≈0.95).

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Generated with Claude Code

…e B2

Pin the absorbing LPDiD estimator against the method authors' own R recipes
(danielegirardi/lpdid) with an alexCardazzi/lpdid cross-check gate:
- benchmarks/R/generate_lpdid_golden.R: in-R panel (+ interior-gap unit) and 6
  variants (variance-weighted, reweight, pmd, direct-covariate, pooled,
  RA-point); writes committed lpdid_test_panel.csv + lpdid_golden.json.
- tests/test_methodology_lpdid.py: skip-guarded parity (att/se to ~1e-12,
  cross-platform asserted at 1e-6/1e-7).
- benchmarks/python/coverage_lpdid_ra.py + lpdid_ra_coverage.json: ungated
  Monte-Carlo study validating the RA influence-function SE calibration (~0.95).

Resolves the two provisional REGISTRY deviation notes in the library's favour
with no estimator change: the RA SE matches the Stata teffects convention
(point-anchored, SE pinned + coverage-validated; no R-package analogue), and the
pooled estimand matches the authors' fixed-composition recipe (correcting the
prior "horizon-stacked" wording). no_composition documented as more paper-faithful
than the R packages (B1-tested). Ticks the REGISTRY B2 checklist box.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Overall Assessment

Looks good — no unmitigated P0/P1 findings.

Executive Summary

  • No estimator code changes; this PR adds LPDiD validation harnesses, goldens, benchmark evidence, and methodology documentation.
  • Methodology deviations for RA SE, pooled semantics, and no_composition are explicitly documented in REGISTRY.md; they are not blockers under the review rules.
  • The new parity tests cover VW, reweight, PMD, direct covariates, pooled, RA point, and pinned RA SE paths.
  • The R generator fails closed on the pinned alexCardazzi/lpdid SHA and cross-checks comparable event-study variants.
  • I could not run the tests locally because pytest and pandas are not installed in this environment.

Methodology

  • Severity: P3 informational
    Impact: RA standard errors still have no runnable R reference, but this is explicitly documented and tracked. The PR anchors RA points to R, pins RA IF SEs, and adds MC coverage evidence.
    Concrete fix: No approval-blocking fix. Continue the tracked follow-up to add a trusted external RA SE anchor.
    Location: docs/methodology/REGISTRY.md:L1847, TODO.md:L119, tests/test_methodology_lpdid.py:L208-L228

  • Severity: P3 informational
    Impact: no_composition is intentionally not R-parity gated because the registry documents it as a deviation from R.
    Concrete fix: None required.
    Location: docs/methodology/REGISTRY.md:L1852, benchmarks/R/generate_lpdid_golden.R:L263-L271

Code Quality

  • Severity: N/A
    Impact: No findings. The changed Python/R harness code is scoped and uses fail-closed checks for missing or mismatched R cross-checks. No new inline inference anti-patterns found.
    Concrete fix: None.

Performance

  • Severity: N/A
    Impact: No findings. The MC coverage study is under benchmarks/ and explicitly ungated.
    Concrete fix: None.

Maintainability

  • Severity: N/A
    Impact: No findings. RA pins include refresh instructions, and golden generation metadata records the R/package versions and pinned commit.
    Concrete fix: None.

Tech Debt

  • Severity: P3 informational
    Impact: The remaining external-anchor gap for RA SE is properly tracked in TODO.md, so it is not a blocker.
    Concrete fix: None for this PR.
    Location: TODO.md:L119

Security

  • Severity: N/A
    Impact: No secrets or sensitive data found in the changed files.
    Concrete fix: None.

Documentation / Tests

  • Severity: P3 informational
    Impact: Added tests cover the key LPDiD parity surfaces and include NaN-consistency handling for missing golden horizons. Local execution was not possible because pytest/pandas are unavailable here.
    Concrete fix: Ensure CI runs tests/test_methodology_lpdid.py; no PR change required based on this review.
    Location: tests/test_methodology_lpdid.py:L103-L228

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jun 29, 2026
@igerber igerber merged commit 48e1f4c into main Jun 29, 2026
31 of 32 checks passed
@igerber igerber deleted the feature/lpdid-r-parity branch June 29, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant