Skip to content

fix(groupby): align groupers by label, raise on mismatch (#827)#830

Merged
FabianHofmann merged 1 commit into
masterfrom
fix-827-groupby-grouper-alignment
Jul 16, 2026
Merged

fix(groupby): align groupers by label, raise on mismatch (#827)#830
FabianHofmann merged 1 commit into
masterfrom
fix-827-groupby-grouper-alignment

Conversation

@FBumann

@FBumann FBumann commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Closes #827.

@FabianHofmann We decided to NOT reindex. As all groupers should behave similarly, they now all raise if the grouper is misaligned.
But Dataframes were reindex before. We could also change this to reindex all groupers instead of raising if its a pure reorder. Users relying on the reindex will get an error now...

Warning

Potentially breaking. Groupers that were previously accepted (silently regrouping by position, or — for DataFrame groupers — silently reindexing) now raise ValueError. Code that relied on a reordered grouper "just working" must reorder the grouper to match the expression's coordinates.

Note

The following content was generated by AI.

What & why

On the default (fast) path, groupby(...).sum() matched a pd.Series/DataArray grouper to the expression by position, ignoring labels — a reordered grouper silently regrouped and produced a wrong result (#827). The three grouper paths also disagreed: DataFrame reindexed by label, the fallback raised.

This makes all grouper types follow one rule, per the issue decision ("Don't reindex, just check size and index fits — else raise"): match by label, never reindex, raise on mismatch.

  • New _check_grouper_alignment(group, data) validates each shared dimension's labels against the expression's coordinates and raises a clear ValueError, distinguishing "same labels in a different order" from "a different set of labels". Runs on both the fast path and use_fallback=True (replacing the fallback's misleading "length does not match" message).
  • Unified across pd.Series, pd.DataFrame and 1-D/N-D DataArray. A grouper without an index along a dim has nothing to align by and keeps the positional match.
  • _encode_multikey_group no longer reindexes (alignment is now guaranteed upstream).

Breaking change

DataFrame groupers previously reindexed a reordered index silently; they now raise like the others. test_linear_expression_groupby_with_dataframe_non_aligned was updated to assert the raise.

Tests

New TestGroupbyGrouperAlignment: reordered Series/DataFrame/DataArray raise, N-D reorder raises, mismatched label-set raises, aligned + unindexed groupers still work (parametrized over use_fallback). Full test_linear_expression.py and all -k group tests pass; ruff and mypy clean.

groupby(...).sum() matched pd.Series/DataArray groupers to the
expression by position, so a grouper whose labels were reordered
silently regrouped. Validate the grouper's labels against the
expression's coordinates and raise on a reorder or a different label
set, unified across Series, DataFrame and DataArray groupers. The
DataFrame path no longer reindexes; all grouper types now follow the
same label-alignment rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codspeed-hq

codspeed-hq Bot commented Jul 15, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 173 untouched benchmarks
⏩ 173 skipped benchmarks1


Comparing fix-827-groupby-grouper-alignment (904ba0a) with master (c02f5db)

Open in CodSpeed

Footnotes

  1. 173 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@FBumann
FBumann requested a review from FabianHofmann July 15, 2026 14:04
@FabianHofmann

Copy link
Copy Markdown
Collaborator

@FBumann one thought that came to my mind. since here we don't have the ability to set the join convention - in contrast to x.mul(y, join="...") I wonder if our choice of "exact" join is the correct choice here. I will take a look at this again tomorrow morning.

@FabianHofmann
FabianHofmann merged commit f3e30ae into master Jul 16, 2026
23 checks passed
@FabianHofmann
FabianHofmann deleted the fix-827-groupby-grouper-alignment branch July 16, 2026 08:08
FBumann added a commit that referenced this pull request Jul 16, 2026
Two follow-ups to the §8-exact work (#831/#834), from an open-items.md audit.

Gap 1 — transition-surface hole: after #831 made a pure reorder raise under v1,
reordered *constant* operands (`x + array`, `-`, `*`, `/`, rhs) were still
silently reindexed by label under legacy with no warning, breaking the
"no silent change" guarantee (coeff and merge already warned). Thread a
`warn_reorder` flag broadcast_to_coords -> _reindex_reordered_dims, set at the
arithmetic const/rhs sites; legacy now emits `_legacy_const_reorder_message`
(accurately: reindexed by label, not positional). Legacy result unchanged;
no double-warn; construction (bounds/mask, strict=True) stays silent.

Groupers — add a strict-alignment paragraph to convention.md §13: a groupby
grouper aligns to the grouped dimension by §8 (reorder or set-mismatch raises,
never positional), multi-key -> flat group dim + aux coords. Implementation is
#830 (on master, lands here on merge). open-items.md records the §8-exact
decision and the pending grouper landing.

Full suite green; ruff/mypy clean.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FBumann added a commit that referenced this pull request Jul 16, 2026
Add a Features entry announcing v1 (`options["semantics"] = "v1"`, legacy stays
the default) with the strict-alignment / user-NaN / absence / aux-coord /
MultiIndex summary and a link to the convention. Mark the changelog and
grouper-landing (#830 merged) items done in open-items.md.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

groupby fast path matches Series/DataArray groupers by position, not by label

2 participants