Skip to content

Return plain CCDData from array API wrappers - #953

Merged
mwcraig merged 3 commits into
astropy:mainfrom
nomad3:fix-927-return-plain-ccddata
Jul 26, 2026
Merged

Return plain CCDData from array API wrappers#953
mwcraig merged 3 commits into
astropy:mainfrom
nomad3:fix-927-return-plain-ccddata

Conversation

@nomad3

@nomad3 nomad3 commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • make _unwrap_ccddata_for_array_api validate inputs before accessing uncertainty
  • unwrap private uncertainty wrappers even when arithmetic has already produced a plain CCDData
  • convert _CCDDataWrapperForArrayAPI results to concrete CCDData while preserving ordinary CCDData subclasses
  • add focused public and helper-level regressions across the supported uncertainty types

Fixes #927

Validation

  • python -m pytest ccdproc -q — 339 passed, 29 skipped
  • focused regression module — 8 passed with NumPy, Dask, and JAX
  • focused regression module with array-api-strict — 7 passed, 1 documented xfail for the pre-existing Astropy uncertainty-propagation boundary
  • Ruff, Black, and git diff --check — passed

Checklist

  • For new contributors: Did you add yourself to the AUTHORS.rst file? (The entry is already proposed in draft Exclude masked weights from average combinations #952, so it is intentionally not duplicated here.)
  • For documentation changes: Does your commit message include a [skip ci]? (Not a documentation-only change.)
  • For bugfixes: Did you add an entry to the CHANGES.rst file?
  • For bugfixes: Did you add a regression test?
  • For bugfixes: Does the commit message include Fixes #927?
  • Does this PR add, rename, move or remove any existing functions or parameters?

AI assistance disclosure

OpenAI Codex was used for source inspection, test design, implementation, and automated review. This PR remains a draft pending the contributor's personal review; the contributor will take responsibility for the final contribution before marking it ready.

@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.63%. Comparing base (b932212) to head (e2d43f3).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #953   +/-   ##
=======================================
  Coverage   96.63%   96.63%           
=======================================
  Files           8        8           
  Lines        1574     1575    +1     
=======================================
+ Hits         1521     1522    +1     
  Misses         53       53           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the Array API wrapper unwrapping path so public ccdproc functions no longer return the private _CCDDataWrapperForArrayAPI type, and so uncertainty wrappers are converted back to Astropy’s public uncertainty classes even when arithmetic already produced a plain CCDData.

Changes:

  • Make _unwrap_ccddata_for_array_api validate the input type before touching .uncertainty, and convert _CCDDataWrapperForArrayAPI instances back to a concrete CCDData.
  • Unwrap private uncertainty wrapper types (_StdDevUncertaintyWrapper, _VarianceUncertaintyWrapper, _InverseVarianceWrapper) even when the container is already a plain CCDData.
  • Add targeted regression tests covering wrapper-copy paths, arithmetic paths, and identity behavior for CCDData subclasses.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
CHANGES.rst Adds a 2.6.0 bugfix entry documenting the corrected Array API wrapper/uncertainty return types.
ccdproc/_ccddata_wrapper_for_array_api.py Fixes _unwrap_ccddata_for_array_api ordering/validation so wrappers are actually converted back and wrapped uncertainties are normalized.
ccdproc/tests/test_ccddata_wrapper_for_array_api.py Adds focused regressions ensuring public APIs return plain CCDData and public uncertainty types across supported backends.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for digging into #927 -- the diagnosis is right, and the reordering here is a real fix for the wrapper leaking out of public functions (on main, the early isinstance(ccd, CCDData) return made the CCDData(ccd) conversion unreachable, since the wrapper is a CCDData subclass). The test coverage is also very welcome.

Two requests:

  1. The conversion itself needs a different mechanism -- see the inline comment. Copy-constructing through CCDData(ccd) coerces the mask to numpy via astropy's setter, which is what's failing the dask jobs on this PR. Swapping __class__ in place fixes the type identity without disturbing any of the backend arrays.
  2. Please rebase onto current main and mark the PR ready for review. main has moved since this branch was cut (#952, #925, and #961 have merged, including changes to CHANGES.rst that will conflict trivially), and CI here is a month stale. Once it's rebased with the fix above, the dask jobs should go green and I'm happy to take a final look.

ccd.uncertainty = _unwrap_uncertainty(ccd.uncertainty)

if isinstance(ccd, _CCDDataWrapperForArrayAPI):
return CCDData(ccd)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the line that's breaking the dask CI jobs (ubuntu-py312-dask-escape-baseline and macos-py312-dask both fail test_transform_image[True-True] with TypeError: Multiple namespaces for array inputs).

CCDData(ccd) re-runs the astropy constructor, and astropy's NDDataArray.mask setter coerces the mask with np.asarray(value, dtype=np.bool_). So for a dask- or jax-backed image, data and uncertainty.array survive in the backend library but mask comes back as a plain numpy array -- reintroducing exactly the coercion _CCDDataWrapperForArrayAPI exists to prevent (its own mask setter uses xp.asarray from the data's namespace instead).

Since the wrapper is a plain-Python subclass of CCDData with no extra state, you can convert it without round-tripping through the constructor:

Suggested change
return CCDData(ccd)
# Avoid CCDData(ccd): astropy's mask setter coerces the mask with
# np.asarray(..., dtype=bool), which would pull non-numpy masks
# back to numpy.
ccd.__class__ = CCDData
return ccd

There's no aliasing concern -- _wrap_ccddata_for_array_api creates a new wrapper object, so mutating it here never touches a caller-owned CCDData. I've verified this variant locally: the full suite passes under numpy (339), dask (334), and jax with x64 (spot-checked, including the previously failing transform test), and after a wrap->unwrap round trip under dask, data, mask, and uncertainty.array all stay dask arrays while type(result) is CCDData.

nomad3 added 3 commits July 26, 2026 14:23
Fixes astropy#927

Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
@nomad3
nomad3 force-pushed the fix-927-return-plain-ccddata branch from 65e98c5 to e2d43f3 Compare July 26, 2026 18:33
@nomad3
nomad3 marked this pull request as ready for review July 26, 2026 18:33
@nomad3

nomad3 commented Jul 26, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the requested in-place change, rebased on main, and reran the relevant tests. Ready for another look—thanks!

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nomad3!

@mwcraig
mwcraig merged commit 1106aa1 into astropy:main Jul 26, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Array API: _unwrap_ccddata_for_array_api never converts the wrapper back to CCDData

3 participants