Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [2.8.3] - 2026-04-02

### Added
- **Silent operation warnings** — 8 operations that previously altered analysis results without informing the user now emit `UserWarning`:
- TROP lstsq → pseudo-inverse numerical fallback
- TwoStageDiD NaN masking of unidentified fixed effects (zeroed out with treatment indicator)
- TwoStageDiD always-treated unit removal (sample size change)
- CallawaySantAnna silent (g,t) pair skipping (zero treated or control observations)
- TROP missing treatment indicator fill with 0 (control)
- Rust → Python backend fallback (previously debug log only)
- Survey weight normalization (pweights/aweights rescaled to mean=1)
- `np.inf` → 0 never-treated convention conversion
- **ImputationDiD pre-period event study coefficients** — pre-treatment "effects" (should be ~0 under parallel trends) for visual pre-trends assessment, following BJS (2024) Test 1
- **TwoStageDiD pre-period event study coefficients** — same pre-trends extension
- **Replicate weight expansion** to 7 additional estimators: DifferenceInDifferences, TwoWayFixedEffects, MultiPeriodDiD, SunAbraham, StackedDiD, ImputationDiD, TwoStageDiD (coverage: 4/13 → 11/13)

### Changed
- ImputationDiD pre-period coefficients use BJS Test 1 (impute Y(0) for treated units in pre-treatment periods)
- SunAbraham replicate weights use full interaction-weighted refit per replicate with cohort-level SEs

### Fixed
- Fix zero-weight demeaning safety in replicate weight paths
- Fix `df_survey` writeback for rank-deficient replicate designs (df=0)
- Fix ImputationDiD `balance_e` zero-qualifying-cohort fallback in pretrends path
- Fix survey zero-mass (g,t) skip warning gap
- Fix SunAbraham positional assignment in replicate loop

## [2.8.2] - 2026-04-02

### Added
Expand Down Expand Up @@ -1085,6 +1112,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `to_dict()` and `to_dataframe()` export methods
- `is_significant` and `significance_stars` properties

[2.8.3]: https://github.com/igerber/diff-diff/compare/v2.8.2...v2.8.3
[2.8.2]: https://github.com/igerber/diff-diff/compare/v2.8.1...v2.8.2
[2.8.1]: https://github.com/igerber/diff-diff/compare/v2.8.0...v2.8.1
[2.8.0]: https://github.com/igerber/diff-diff/compare/v2.7.6...v2.8.0
Expand Down
2 changes: 1 addition & 1 deletion diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@
Bacon = BaconDecomposition
EDiD = EfficientDiD

__version__ = "2.8.2"
__version__ = "2.8.3"
__all__ = [
# Estimators
"DifferenceInDifferences",
Expand Down
2 changes: 1 addition & 1 deletion docs/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> A Python library for Difference-in-Differences causal inference analysis. Provides sklearn-like estimators with statsmodels-style output for econometric analysis.

- Version: 2.8.2
- Version: 2.8.3
- Repository: https://github.com/igerber/diff-diff
- License: MIT
- Dependencies: numpy, pandas, scipy (no statsmodels dependency)
Expand Down
100 changes: 99 additions & 1 deletion docs/survey-roadmap.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Survey Data Support Roadmap

This document captures the survey data support roadmap for diff-diff.
All phases (1-6) are implemented.
Phases 1-7 are implemented. Phase 8 (maturity refinements) is planned.

## Implemented (Phases 1-2)

Expand Down Expand Up @@ -202,3 +202,101 @@ variance estimation for staggered triple differences.

**Reference:** Ortiz-Villavicencio, M. & Sant'Anna, P.H.C. (2025).
"Better Understanding Triple Differences Estimators." arXiv:2505.09942.

---

## Phase 8: Survey Maturity

Refinements to close remaining gaps versus R's `survey` package and improve
practitioner experience. Prioritized by user impact.

### 8a. Successive Difference Replication (SDR)

**Priority: High.** ACS PUMS — the most common US survey dataset for DiD
policy evaluation — provides 80 SDR replicate weight columns. Without SDR
support, these users can't use their provided replicate weights directly.

**What's needed:**
- Add `"SDR"` to `valid_rep_methods` in `SurveyDesign`
- Variance formula: `V = 4/R * sum((theta_r - theta)^2)` — a scaling
difference from BRR, not a new algorithm
- Wire through `compute_replicate_vcov()` and `compute_replicate_if_variance()`

**Reference:** Fay, R.E. & Train, G.F. (1995). "Aspects of Survey and
Model-Based Postcensal Estimation of Income and Poverty Characteristics
for States and Counties." ASA Proceedings.

### 8b. FPC in ImputationDiD and TwoStageDiD

**Priority: High.** Both estimators now support replicate weights and TSL
with strata/PSU, but reject FPC outright (`NotImplementedError`). Adding
FPC is incremental — thread `fpc` through the existing TSL variance path.
Matters for finite population surveys (common in state-level sampling).

**Current gate:** `imputation.py:280`, `two_stage.py:268`

### 8c. Silent Operation Warnings

**Priority: High.** Add `UserWarning` emissions for operations that
silently alter analysis results:
- TROP lstsq → pseudo-inverse numerical fallback
- TwoStageDiD NaN masking of unidentified fixed effects
- TwoStageDiD always-treated unit removal
- CallawaySantAnna silent (g,t) pair skipping
- TROP missing treatment indicator fill with 0
- Rust → Python backend fallback (currently debug log only)
- Survey weight normalization (pweights rescaled to mean=1)
- `np.inf` → 0 never-treated conversion

### 8d. Lonely PSU "adjust" in Bootstrap

**Priority: Medium.** `lonely_psu="adjust"` works for analytical (TSL)
variance but raises `NotImplementedError` for survey-aware bootstrap
(2 raises in `bootstrap_utils.py`). Real survey data regularly has
singleton strata. Users needing bootstrap inference with such data hit
a wall.

**Reference:** Rust, K.F. & Rao, J.N.K. (1996). "Variance Estimation
for Complex Surveys Using Replication Techniques." Statistical Methods
in Medical Research 5(3).

### 8e. Survey Diagnostics and Utilities

**Priority: Medium.** Small additions that signal maturity to survey
statisticians:
- **CV on estimates**: coefficient of variation (SE/estimate) on results
objects — trivial to add, used by federal agencies for publication
standards (NCHS requires CV < 30% for releasable estimates)
- **Weight trimming**: `trim_weights(data, weight_col, upper=None,
quantile=None)` utility in `prep.py` for capping extreme weights
- **ImputationDiD pretrends + survey**: pre-trends F-test currently
ignores survey variance (`NotImplementedError` at `imputation.py:240`)

### 8f. Survey Compatibility Matrix

**Priority: Medium.** Users discover survey support limits by hitting
`NotImplementedError` at runtime. Add a table to the survey tutorial
or `choosing_estimator.rst` showing which estimator × survey feature
combinations are supported (weights, strata/PSU, FPC, replicate weights,
bootstrap + survey).

### 8g. Documentation-Only Items

**Priority: Low.** No code changes required:
- **Multi-stage design**: document that single-stage (strata + PSU)
is sufficient for variance estimation per Lumley (2004) Section 2.2.
Don't implement multi-stage — it adds complexity without changing
results for DiD applications.
- **Post-stratification / calibration**: document that `SurveyDesign`
expects pre-calibrated weights. Point users to `samplics` or R's
`survey::calibrate()` for weight calibration. This is data prep,
not DiD estimation — out of scope.

### Deferred

| Estimator | Capability | Reason |
|-----------|-----------|--------|
| SyntheticDiD | Replicate weights | No published theory on replicate weights + unit weight optimization |
| TROP | Replicate weights | No published theory on replicate weights + nuclear norm regularization |
| BaconDecomposition | Replicate weights | Diagnostic tool with no inference — replicate weights don't apply |
| EfficientDiD | Covariates + survey, cluster + survey, bootstrap + survey | Lower demand, newer estimator; 3 `NotImplementedError` paths |
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "maturin"

[project]
name = "diff-diff"
version = "2.8.2"
version = "2.8.3"
description = "Difference-in-Differences causal inference with sklearn-like API. Callaway-Sant'Anna, Synthetic DiD, Honest DiD, event studies, parallel trends."
readme = "README.md"
license = "MIT"
Expand Down
2 changes: 1 addition & 1 deletion rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "diff_diff_rust"
version = "2.8.2"
version = "2.8.3"
edition = "2021"
description = "Rust backend for diff-diff DiD library"
license = "MIT"
Expand Down
Loading