Compute Combiner.clip_extrema mask with rank comparison instead of scatter - #994
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #994 +/- ##
==========================================
- Coverage 97.75% 97.75% -0.01%
==========================================
Files 9 9
Lines 1785 1780 -5
==========================================
- Hits 1745 1740 -5
Misses 40 40
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| ] | ||
|
|
||
| def _clip_extrema_combiner(data): | ||
| if xp.__name__ == "array_api_strict": |
There was a problem hiding this comment.
Why do we still need this guard? I thought the point of this PR was to fix clip_extrema
There was a problem hiding this comment.
We don't — it's stale. The guard was carried over from the #928 test (a4ffcef, 2026-07-19), when Combiner.__init__ still built the stack with a nested xp.asarray([...]) that array-api-strict rejects. That was fixed by xp.stack in ebfe8de (2026-08-22, the #965 item), so the branch has been dead since; this PR only preserved it because the plan said to reuse the old construction verbatim.
Checked on this branch with the same three images, on both strict's default device and device1:
Combiner([CCDData(xp.asarray(im, device=...), unit=u.adu) for im in data]).clip_extrema(nlow=1, nhigh=1)
__init__ and clip_extrema both succeed and the mask stays in array_api_strict on the input device. test_clip_extrema_3d and friends already construct their Combiner the normal way on strict and pass.
So _clip_extrema_combiner can go entirely; each test just does
c = Combiner([CCDData(xp.asarray(image, device=xp_device), unit=u.adu) for image in data])(device=xp_device so the device assertion in the first test still exercises the non-default device on strict.) Happy to push that.
— Written by Claude at @mwcraig's direction.
…atter clip_extrema scattered into the mask through per-pixel integer fancy indexing (xpx.at(...)[flat_index].set(True)), the only place in the codebase doing that. array_api_strict rejects integer-array __setitem__, and the array API standard has no put_along_axis equivalent. A second argsort along axis 0 inverts the first: ranks[k, ...] is the rank of image k's value at each pixel, so comparing ranks against nlow/nhigh directly produces the same mask without any scatter. Both argsort calls default to stable=True, so tie-breaking is unchanged from the old code. Verified against a reference implementation across 8400 random trials (with ties and NaNs) on top of the new code path. Replace test_clip_extrema_keeps_indices_in_array_namespace, which existed to spy on the now-removed scatter's indices, with two tests that check the namespace/device of the resulting mask and its content directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
The object.__new__(Combiner) branch for array-api-strict dated from before Combiner.__init__ used xp.stack; it has been unnecessary since then, so construct the combiner the normal way on every backend. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
d414ddc to
f61e08e
Compare
Part of #971 (the clip_extrema bucket).
array_api_strictstrict test run on top ofmain(711bb26): 41 failed → 37 failed, no new failures on any of the four backends (numpy, jax, dask, strict). The failure list after this change is identical tomain's minus the fourclip_extrematests.Combiner.clip_extremamasked thenlowlowest andnhighhighest per-pixel values by building flat integer indices by hand and scattering into the mask withxpx.at(...)[flat_index].set(True)-- the only integer-arrayxpx.at(...).setcall anywhere in the codebase.array_api_strictrejects fancy-indexed__setitem__, and there is noput_along_axisin the array API standard, so this raisedIndexErroron strict.A second
argsortalong axis 0 inverts the first:ranks[k, ...]is the rank of image k value at each pixel, so comparing ranks againstnlow/nhighproduces the identical mask without any scatter. Bothargsortcalls default tostable=True, so tie-breaking is unchanged from the old code -- verified against a hand-written reference implementation across 8400 random trials (with ties and NaNs, several shapes) run against the new code path, 0 mismatches. This also let thenumpy.mgrid-based coordinate construction and the now-unusedfrom numpy import mgrid as np_mgridimport be deleted.test_clip_extrema_keeps_indices_in_array_namespace, which existed to spy on the removed scatter indices via a monkeypatchedxpx.at, no longer has anything to spy on. It is replaced with two tests: one checking the resulting mask namespace/device (reusing the same tied(3, 2, 3)fixture data and expected mask), and one checking the mask content directly against a documented 3-way tie.The four target strict failures (
test_clip_extrema_3d,_alone,_via_combine,_with_other_rejection) all pass; numpy, jax (JAX_ENABLE_X64=True), and dask are unaffected (same pass/skip/xfail counts asmain, no failures). Ruff clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME