docs: add CiC/QDiD paper reviews (Athey-Imbens 2006, Callaway-Li-Oka 2018, Melly-Santangelo 2015, Ciaccio 2024)

[igerber]

Summary

• PR-A of the Changes-in-Changes / distributional DiD initiative (2-PR workflow: reviews merge first, implementation follows in PR-B against the reviewed scope)
• docs/methodology/papers/athey-imbens-2006-review.md (413 lines): the core source, reviewed from the published Econometrica 74(2) typeset. Covers CiC identification (Theorem 3.1 / eq. 9), the exact inf-based and sup-based inverse-CDF conventions where implementations diverge (eqs. 8, 24, 34-35, A.1-A.2), the plug-in ATT/quantile estimators (eqs. 36, 17-18), the QDiD variant (eq. 22 + p. 447 transformation), influence-function inference (Theorems 5.1-5.7), discrete-outcome bounds (Section 4/5.2), covariate routes, and the Section 6 multi-group machinery with Appendix B's 26-case covariance rule
• docs/methodology/papers/callaway-li-oka-2018-review.md (417 lines): two-period panel QTT under Distributional DiD + Copula Invariance (arXiv:1702.03618v1, pinned; JoE 206(2) is the version of record). Reviewed as panel-mode grounding and qte-package foundations; carries the exchangeable-bootstrap validity machinery relevant to v1's bootstrap-first inference
• docs/methodology/papers/melly-santangelo-2015-review.md (522 lines): CiC with covariates via quantile regression (October 2015 working paper, pinned; still unpublished). Documents the QR pipeline blueprint and the DiD-bias motivation; flags draft typos verbatim
• docs/methodology/papers/ciaccio-2024-review.md (619 lines): staggered/multi-period distributional DiD (arXiv:2408.01208v2, pinned). Un-parks the ROADMAP "Distributional DiD for staggered timing" row to reviewed-deferred; documents the Unc-vs-Cond Monte Carlo bias evidence backing the covariates-deferral documentation
• v1 scope decisions (2026-07-12) recorded inside each review: ship ChangesInChanges + QDiD (2x2, continuous outcomes, bootstrap inference, panel + repeated cross-section modes, qte::CiC()/qte::QDiD() golden-file parity); defer covariates, discrete bounds, analytical SEs, and staggered extension - reviewed here so every deferral is documented, not silent

Methodology references (required if estimator / math changes)

• Method name(s): ChangesInChanges (CiC) + QDiD (planned v1 estimators); PanelQTT (Callaway-Li-Oka), CiC with covariates (Melly-Santangelo), staggered distributional DiD (Ciaccio) - reviewed, implementation deferred
• Paper / source link(s):
  • Athey & Imbens (2006), Econometrica 74(2), 431-497. https://doi.org/10.1111/j.1468-0262.2006.00668.x
  • Callaway, Li & Oka (2018), Journal of Econometrics 206(2), 395-413. https://doi.org/10.1016/j.jeconom.2018.06.008 (reviewed from arXiv:1702.03618v1)
  • Melly & Santangelo (2015), working paper (October 2015 draft), Bern University / EC-JRC
  • Ciaccio, arXiv:2408.01208v2
• Any intentional deviations from the source (and why): None - documentation-only PR. Library-convention choices vs the papers (e.g. bootstrap-first inference where Athey-Imbens is analytical-only) are explicitly flagged inside the reviews as conventions to document at implementation time, not silent deviations
• REGISTRY.md is intentionally untouched: registry entries land with the implementation in PR-B (each review states this in its header)

Validation

• Tests added/updated: No test changes (documentation-only PR)
• Backtest / simulation / notebook evidence (if applicable): N/A

Security / privacy

• Confirm no secrets/PII in this PR: Yes (PDFs remain local-only under gitignored papers/)

---

Generated with Claude Code

🤖 Generated with Claude Code

https://claude.ai/code/session_011hismLBFcUbUzvDRq8ruWb

[github-actions]

Overall Assessment

✅ Looks good — no unmitigated P0/P1 findings. The PR is documentation-only and does not change estimator code, inference behavior, defaults, or public APIs. I found several P2/P3 documentation issues that should be cleaned up before these notes become implementation guidance.

Executive Summary

• No live statistical-output, NaN/Inf, control-group, weighting, or variance code paths changed.
• The new reviews are generally careful about deferrals and unresolved source-paper ambiguities.
• A few formulas/source mappings are misleading enough to matter for future implementation work.
• The ROADMAP status claimed by the new Ciaccio review is not actually updated in ROADMAP.md.
• No security or secret exposure found in the changed markdown.

Methodology

Finding 1

Severity: P2
Location: docs/methodology/papers/athey-imbens-2006-review.md:L322-L348, docs/methodology/papers/ciaccio-2024-review.md:L529-L531
Impact: The Athey-Imbens review groups “Ciaccio / ecic” together as the reference for the deferred multi-group/period CiC extension, but the Ciaccio review correctly says Ciaccio is a copula-based Callaway-Li-style staggered QTT method and explicitly warns not to conflate it with ecic, which is staggered event-study CiC. This could send a future implementer to the wrong identification strategy or parity target.
Concrete fix: In the Athey-Imbens review, separate these threads: Athey-Imbens Section 6 / ecic for staggered or multi-group CiC; Ciaccio for a distinct staggered distributional DiD/QTT estimator.

Finding 2

Severity: P2
Location: docs/methodology/papers/ciaccio-2024-review.md:L251-L259
Impact: The IPW estimator display is labeled as F^{p,-1} while the prose says it is a reweighted CDF, and the summation is written over i in r while multiplying by the never-treated indicator C. If copied literally, this produces zero weights for a treated-cohort-only sum. The prose partially mitigates this, but the formula is still dangerous as implementation guidance.
Concrete fix: Add a Gaps/Uncertainties note for the sum_{i in r} + C inconsistency, and rewrite the implementation checklist formula as a Hájek-normalized CDF over the never-treated/all-sample observations using C_i * p_r(X_i) / (p_r * (1 - p_r(X_i))), then invert it.

Finding 3

Severity: P2
Location: docs/methodology/papers/athey-imbens-2006-review.md:L170-L174
Impact: The analytical-SE note calls Y_tilde_gt the support midpoint but defines it as (y_upper_gt - y_lower_gt)/2, which is half the range, not the midpoint unless the lower endpoint is zero. If later copied into the deferred analytical variance path, the one-sided density switch can be wrong near boundaries.
Concrete fix: Verify the paper’s footnote 31 expression and either change this to (y_upper_gt + y_lower_gt)/2 or explicitly document the source text if it is an as-printed typo requiring correction before implementation.

Finding 4

Severity: P2
Location: docs/methodology/papers/melly-santangelo-2015-review.md:L285-L289
Impact: The derivative for phi^{PP}(F, G) = F^{-1} o G shows a negative sign on the h_2 perturbation term. By the standard inverse-map derivative, the perturbation through G should enter with a positive sign, unless the paper uses an unusual sign convention. This matters for any future analytical covariance derivation.
Concrete fix: Re-check the source PDF and either correct the sign or add it to the suspected-typos list with an instruction to re-derive the derivative before analytical SE implementation.

Code Quality

No findings. The PR adds markdown only; no Python/Rust/R code quality issues are in scope.

Performance

No findings. No executable path or benchmark-sensitive code changed.

Maintainability

Finding 5

Severity: P3
Location: docs/methodology/papers/ciaccio-2024-review.md:L16-L16, docs/methodology/papers/ciaccio-2024-review.md:L553-L553, ROADMAP.md:L35-L35, ROADMAP.md:L87-L91
Impact: The new Ciaccio review says the ROADMAP row has moved from “commit when a user reports need” to “reviewed, implementation deferred pending demand,” but ROADMAP.md still contains the old wording. This creates two competing sources of truth.
Concrete fix: Update ROADMAP.md to the reviewed-deferred status, or remove the “row moves” claim from the paper review.

Tech Debt

No blocker findings. The docs clearly mark covariates, discrete bounds, analytical SEs, and staggered extensions as deferred. Because no implementation is added in this PR, lack of TODO.md entries is not a release blocker.

Security

No findings. The diff contains paper-review markdown only; no credentials, tokens, private data, or workflow execution changes.

Documentation/Tests

Finding 6

Severity: P3
Location: docs/methodology/papers/athey-imbens-2006-review.md:L211-L211
Impact: Minor markdown/readability issue: `F_hat^{-1}(0) = ` sample minimum closes the inline code before “sample minimum,” making the convention harder to scan.
Concrete fix: Change to `F_hat^{-1}(0) = sample minimum` or equivalent prose.

No tests are needed for this documentation-only PR.

[github-actions]

🔁 AI review rerun (requested by @igerber)

Head SHA: ce5972ca057f70e4b4cd098b37ab187ef8f71d49

---

Overall Assessment

✅ Looks good — no unmitigated P0/P1 findings. This re-review found the previous issues addressed, and the PR remains documentation-only with no estimator, inference, default, or public API changes.

Executive Summary

• No live statistical or estimator code paths changed.
• Prior methodology-doc issues were resolved: ecic vs Ciaccio is no longer conflated, IPW notation hazards are documented, and suspected source typos are called out.
• ROADMAP.md now matches the Ciaccio review’s reviewed-deferred status at ROADMAP.md:L35.
• No new NaN/Inf inference, weighting, variance, control-group, or parameter-propagation risks were introduced.
• No secrets or security-sensitive changes found in the changed markdown.

Methodology

No findings.

Prior findings are resolved:

• Athey-Imbens now separates Section 6 / ecic from Ciaccio’s copula-based staggered QTT method: docs/methodology/papers/athey-imbens-2006-review.md:L322-L328.
• Ciaccio IPW notation is explicitly treated as an as-printed issue with implementable Hájek-normalized guidance: docs/methodology/papers/ciaccio-2024-review.md:L251-L259, docs/methodology/papers/ciaccio-2024-review.md:L620.
• Athey-Imbens footnote 31 midpoint/half-range issue is documented as a suspected typo: docs/methodology/papers/athey-imbens-2006-review.md:L170-L174, docs/methodology/papers/athey-imbens-2006-review.md:L391.
• Melly-Santangelo PP derivative sign concern is flagged with a re-derive-before-implementation warning: docs/methodology/papers/melly-santangelo-2015-review.md:L285-L291, docs/methodology/papers/melly-santangelo-2015-review.md:L509.

Code Quality

No findings. The PR changes markdown only.

Performance

No findings. No executable or benchmark-sensitive path changed.

Maintainability

No findings. The previous ROADMAP inconsistency is fixed at ROADMAP.md:L35.

Tech Debt

No findings. Deferred scope is explicitly documented in the paper reviews, e.g. Ciaccio not shipping in v1 at docs/methodology/papers/ciaccio-2024-review.md:L16-L21 and Melly-Santangelo covariates deferred at docs/methodology/papers/melly-santangelo-2015-review.md:L16-L20.

Security

No findings. No credentials, tokens, workflow execution changes, or sensitive files are added.

Documentation/Tests

No findings. No tests are needed for this documentation-only PR. The previous minor markdown readability issue around F_hat^{-1}(0) is fixed/readable at docs/methodology/papers/athey-imbens-2006-review.md:L211.