Conversation
Overall AssessmentExecutive Summary
MethodologyP1 — Cited LW paper revisions are not traceable to public SSRN metadataLocation: Impact: The registry and paper reviews claim the transformation paper was reviewed as a June 8, 2026, 61-page SSRN revision and the small-N paper as a February 3, 2026 revision. The public SSRN pages currently show SSRN 4516518 as 52 pages, posted July 27, 2023, last revised December 25, 2025, date written December 24, 2025; and SSRN 5325686 as 36 pages, last revised January 3, 2026, date written January 3, 2026. That makes the new methodology contract non-reproducible against the cited public source. (papers.ssrn.com) Concrete fix: Reconcile all LW citations/review headers/registry dates with the public SSRN metadata, or add a stable archived source for the newer PDFs and explicitly document the SSRN-metadata discrepancy. P1 — WATT aggregation spec loses the unbalanced-panel weighting nuanceLocation: Impact: The registry specifies Concrete fix: Change the registry aggregation bullet and checklist to define Code QualityNo P0/P1 findings. The new binary download helper verifies cached and freshly downloaded bytes before use, re-downloads stale cache entries, and avoids caching checksum-mismatched fresh downloads. PerformanceNo findings. The added code paths are dataset-loading paths and do not affect estimator hot loops. MaintainabilityNo P1 findings. The dataset dispatch and public exports are consistently wired through Tech DebtP3 — Legacy loader fallback alignment is trackedLocation: Impact: The four older loaders still have quieter synthetic fallback behavior than the new loaders, but the PR adds a TODO entry tracking this. Concrete fix: No approval-blocking action required; this is properly tracked deferred work. SecurityNo findings. The new data downloads are HTTP, but every cached/fresh byte load is pinned by SHA-256 before parsing; the RePEc entry also confirms the sample data file URLs and MIT license. (ideas.repec.org) Documentation / TestsP3 — Test execution not verified in this review environmentLocation: Impact: The added tests cover fallback construction, loader dispatch, checksum mismatch, stale-cache replacement, validators, and integration examples, but I could not execute them here because required tooling/dependencies are unavailable. Concrete fix: Run Path to Approval
|
|
Both P1s addressed in 823a756. P1 (SSRN metadata): I re-verified the live SSRN pages today (2026-07-13):
The metadata quoted in the review (52 pages / last revised Dec 25, 2025 / Jan 3, 2026) matches the superseded December 2025 / January 2026 revisions - consistent with a stale cache or mirror of the SSRN pages. Per the suggested fallback fix, provenance is now self-documenting regardless: both paper-review headers pin the exact reviewed artifact (PDF SHA-256 + cover-page date) alongside a live-verified SSRN metadata block with an explicit stale-cache note; DOIs added to REGISTRY.md and references.rst. P1 (WATT weights): Agreed - good catch, and it was already recorded as a nuance in our own review doc. The REGISTRY aggregation bullet and requirements checklist now define P3 (test execution): expected - tests run under the label-gated CI; the 48 tests in |
|
/ai-review |
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment✅ Looks good — no unmitigated P0/P1 findings. Executive Summary
MethodologyP3 — Balanced-panel WATT shorthand remains in secondary docsLocation: Impact: The authoritative registry now correctly uses contributing treated units for Concrete fix: Rephrase these secondary mentions to “contributing-treated-unit weighted, simplifying to cohort-size weights in balanced panels.” Code QualityNo P0/P1 findings. The binary cache helper verifies cached and fresh PerformanceNo findings. The new paths are dataset-loading and validation paths, not estimator hot loops. MaintainabilityP2 — Generated Playwright capture artifacts are committedLocation: Impact: These are browser-session artifacts, not library/docs source. They add noisy, timestamped external-page captures and one binary-ish log file to the repository. Concrete fix: Remove Tech DebtP3 — Legacy loader fallback alignment is trackedLocation: Impact: The older CSV-backed loaders still have quieter fallback behavior than the new loaders, but this PR tracks the limitation explicitly. Concrete fix: No approval-blocking action required. SecurityNo findings. The new HTTP downloads are checksum-pinned before parsing, and the committed textual Playwright logs did not show secrets in the inspected output. Removing the generated artifacts is still recommended under Maintainability. Documentation / TestsP3 — Test execution not verified in this environmentLocation: Impact: Added tests cover fallback structures, source validators, checksum mismatch behavior, stale-cache replacement, cache clearing, dispatch, and estimator integration examples. I could not execute them here because Concrete fix: Run |
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment✅ Looks good — no unmitigated P0/P1 findings. Executive Summary
MethodologyNo P0/P1 findings. The LWDiD registry is documentation-only and explicitly marks implementation requirements as unchecked until the estimator lands. The relevant WATT text is now methodologically sound at Code QualityNo findings. The new binary download helper verifies cached and fresh bytes before PerformanceNo findings. The changed runtime paths are dataset loading, cache I/O, validation, and synthetic fallback generation, not estimator hot paths. MaintainabilityNo findings. The prior generated-artifact issue is addressed by ignoring Tech DebtP3 — Legacy loader fallback alignment is trackedLocation: Impact: The four older dataset loaders still have quieter fallback behavior than Concrete fix: No approval-blocking action required. SecurityNo findings. The HTTP-only Documentation / TestsP3 — Test execution not verified in this environmentLocation: Impact: The added tests cover fallback shape/treatment behavior, source validators, checksum mismatch rejection, stale-cache replacement, cache clearing, dataset dispatch, and estimator integration examples, but I could not execute them here. Concrete fix: Run |
…on rubric) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg
…D REGISTRY entry + references (precursor to #588) Loaders download the MIT-licensed ancillary datasets of the authors' Stata lwdid package (SSC): pinned SHA-256 verification of every byte-load (HTTP-only host), stale-cache re-download, structural validation against source invariants, loud UserWarning + df.attrs['source'] marker on synthetic fallback, seeded local-RNG fallback constructors. REGISTRY.md gains the maintainer-authored LWDiD section (E.1 contributing-unit WATT weights; provenance pinned via reviewed-PDF SHA-256 + live-verified SSRN metadata). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment✅ Looks good — no unmitigated P0/P1 findings. Executive Summary
MethodologyNo findings. The PR adds the Code QualityNo findings. The loader plumbing is complete: public exports at PerformanceNo findings. The added paths are dataset download/cache/validation and small synthetic fallback generation, not estimator hot paths. MaintainabilityNo findings. The source validators are explicit and covered by focused tests for duplicate rows, non-constant cohorts, missing cohorts, and wrong never-treated counts at Tech DebtP3 — Legacy loader fallback alignment is trackedLocation: Impact: Older dataset loaders still have quieter fallback behavior than the new Concrete fix: No approval-blocking action required. SecurityNo findings. HTTP-only binary downloads are guarded by pinned SHA-256 checks before cache write or Stata parsing at Documentation / TestsP3 — Test execution not verified in this environmentLocation: Impact: Added tests cover fallback structures, source validators, checksum mismatch rejection, stale-cache replacement, cache clearing, dispatch, and estimator-smoke paths, but I could not execute them locally because Concrete fix: Run |
…oc-snippet mock loaders with prop99/walmart The doc-snippets harness replaces sys.modules['diff_diff.datasets'] with a mock module; the new loaders' snippets need mock counterparts there. The wrapped Returns bullet in load_prop99 broke the -W docs build (unexpected indentation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall Assessment Executive Summary
Methodology Affected method: Code Quality Loader export and dispatch paths are wired through Performance The added work is download/cache/validation and small synthetic fallback generation, not estimator hot-path code. Maintainability Source validators and fallback tests are explicit at Tech Debt P3 — Legacy loader fallback alignment is trackedLocation: Impact: Older dataset loaders still have quieter fallback behavior than the new Concrete fix: No approval-blocking action required. Security HTTP-only Documentation/Tests P3 — Test execution not verified in this environmentLocation: Impact: Added tests cover fallback structure, source validators, checksum mismatch rejection, stale-cache replacement, cache clearing, dispatch, and estimator smoke paths, but I could not run them here because Concrete fix: Run |
Summary
docs/methodology/papers/lee-wooldridge-{2025,2026}-review.md): the June 8, 2026 revision of the transformation paper (SSRN 4516518) and the February 3, 2026 revision of the small-sample inference paper (SSRN 5325686).## LWDiDREGISTRY.md section (assumptions NAC/CPTC/OVLC + CNAS/CPTS/OVLS + CHT, transformations, RA/IPW/IPWRA, Algorithm 1 multiplier bootstrap, exact-t small-N inference layer, WATT(r) + composite aggregation, edge cases, replication targets) following the LP-DiD precedent of landing the registry entry with the paper review, ahead of the implementation.load_prop99()+load_walmart()dataset loaders: download the MIT-licensed ancillary datasets of the authors' Statalwdidpackage (SSC, Boston College host) - the Prop 99 smoking panel (39 states, single 1989-treated unit) and the Walmart entry county panel (1,277 counties, staggered 1986-1999, 391 never-treated; the current paper revision's sample). Binary download helper with pinned SHA-256 verification on every byte-load (host is HTTP-only), stale-cache re-download, post-download structural validation against source invariants, loudUserWarning+df.attrs["source"]marker on synthetic fallback, seeded local-RNG fallback constructors.docs/api/datasets.rst,index.rstautosummary + tracked_autosummarystubs,llms.txt/llms-full.txtcatalogs,docs/references.rstLee-Wooldridge subsection, CHANGELOG entry for the loaders, TODO row for aligning the four legacy loaders with the loud-fallback pattern.Methodology references (required if estimator / math changes)
lwdid(Hur, Lee & Wooldridge; SSC s459672, MIT)Validation
tests/test_datasets.py(+12 tests, 48 passing): per-dataset fallback structure/treatment/values/size, loud-fallback warning +attrsmarker, checksum-mismatch rejection + stale-cache re-download, per-invariant source-validator rejection tests against a valid-shaped fixture,clear_cache.dtabehavior,load_datasetdispatch, integration fits (DifferenceInDifferences,CallawaySantAnna).Security / privacy
Generated with Claude Code