Skip to content

Combiner.sigma_clipping: low_thresh/high_thresh docstring no longer matches behaviour (None means 3σ, negatives are not abs()-ed) #1002

Description

@mwcraig

The sigma_clipping docstring (ccdproc/combiner.py, currently lines 443–452) still says:

low_thresh : positive float or None, optional
Threshold for rejecting pixels that deviate below the baseline value.
If negative value, then will be convert to a positive value.
If None, no rejection will be done based on low_thresh. Default is 3.

high_thresh : positive float or None, optional
Threshold for rejecting pixels that deviate above the baseline value.
If None, no rejection will be done based on high_thresh. Default is 3.

That describes the original ccdproc implementation. In git show v2.0.1:ccdproc/combiner.py the method does

if low_thresh is not None:
    if low_thresh < 0:
        low_thresh = abs(low_thresh)
    mask = (self.data_arr - baseline < -low_thresh * dev)
    ...
if high_thresh is not None:
    mask = (self.data_arr - baseline > high_thresh * dev)
    ...

so None really did mean "skip that side", and a negative low_thresh (only low_thresh) was made positive.

Since 2.4.0 (5d2abc0, "Always use astropy's sigma clipping and default to string-based functions", #794; the use_astropy option itself arrived in 2.2.0 with d57ca13) the method forwards the thresholds straight to astropy.stats.sigma_clip(..., sigma_lower=low_thresh, sigma_upper=high_thresh, ...). astropy's SigmaClip.__init__ (astropy 8.0.1, astropy/stats/sigma_clipping.py lines 175–176) does

self.sigma_lower = sigma_lower or sigma
self.sigma_upper = sigma_upper or sigma

with sigma=3 by default, so today:

  • low_thresh=None / high_thresh=None clip at on that side rather than not clipping;
  • low_thresh=0 / high_thresh=0 likewise become 3σ instead of rejecting everything off-centre;
  • a negative threshold is passed through unchanged, which turns the comparison inside out (nothing is ever rejected on that side) rather than being converted to a positive value.

None of this is covered by the test suite, and the docstring was not updated in #794.

Related: #1001 (array-API fallback for sigma_clipping) deliberately reproduces the astropy semantics on non-NumPy namespaces so all backends agree, and changes the docstring to say that None is treated as 3; the negative-threshold wording and the 0 case are left as-is there. #1000 is the companion sigma_func change. Both are part of #929.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions