Skip to content

Tests: the redactor scaling check holds on a busy machine (#4788) - #4819

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/4788-redactor-scaling-check
Sep 29, 2026
Merged

erikdarlingdata merged 1 commit into
devfrom
fix/4788-redactor-scaling-check

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Fixes #4788

Why

PgSettingRedactorTests.LongInputWithNoSeparator_ScalesNearLinearlyNotCatastrophically(trailingAssignment: True) failed once in a full Darling.Tests run on a busy machine. The class passed alone right after. The test times PgSettingRedactor.Redact at 2,000 and 32,000 characters (minimum of 5 repeats each) and fails when the larger pass takes 64 times as long as the smaller one, with the smaller time floored at 1 ms. So the bar is about 64 ms for the large pass, and when the rest of a 17,000-test suite keeps the machine busy for all 5 repeats, the minimum can pass 64 ms without any regex problem.

What changes

The first alternative from the issue: the class runs with no other test class running.

  • New PgSettingRedactorTimingCollection: [CollectionDefinition("pg-setting-redactor-timing", DisableParallelization = true)], the same shape as PgFileSettingsStaticsCollection.
  • PgSettingRedactorTests carries [Collection("pg-setting-redactor-timing")], with a doc paragraph saying why.

Nothing else changes. Both assertions stay exactly as they were: the exact-output checks, and the scaling check (bar of 64 times, 1 ms floor on the baseline, 5 repeats, minimum of the repeats). No retry was added and nothing was widened.

The original failure came from a loaded machine and is not reproduced here. This change removes the concurrency it came from and does not loosen the bar, which is what keeps a real quadratic pattern failing (below).

RED plant

The issue asks that a pattern that backtracks badly but finishes under the 100 ms regex timeout still fails the scaling assertion, before and after the change. The plant was a temporary extra pattern in PgSettingRedactor.Redact, (?:pass)+=y (a run of pass with a suffix that never matches, so every start position rescans the rest of the run). It changes no output, so the exact-output checks still pass under it and only the timing check can catch it. It measured 66 to 103 ms at 32,000 characters and under 1.3 ms at 2,000, so at the large size it sits right at the 100 ms match timeout (a timeout is caught inside Redact, which then masks the value, so the measured time stays near 100 ms either way). The plant is removed and is not in the diff.

Before the change (plant in, class not yet in the collection): both theory rows failed.

LongInputWithNoSeparator_ScalesNearLinearlyNotCatastrophically(trailingAssignment: True) [FAIL]
  Expected near-linear scaling (~16x, 32000 chars / 2000 chars) well under quadratic (256x); got 74.2x (89.801ms / 1.210ms).
LongInputWithNoSeparator_ScalesNearLinearlyNotCatastrophically(trailingAssignment: False) [FAIL]
  Expected near-linear scaling (~16x, 32000 chars / 2000 chars) well under quadratic (256x); got 96.5x (96.527ms / 0.753ms).

After the change (plant still in, class in the collection), 4 runs. Each run failed the scaling assertion in one of the two rows:

(trailingAssignment: False) got 102.7x (102.743ms / 0.805ms)
(trailingAssignment: False) got 66.1x (66.108ms / 0.984ms)
(trailingAssignment: False) got 95.8x (95.794ms / 0.801ms)
(trailingAssignment: True)  got 99.6x (99.620ms / 0.676ms)

The other row passed in each of those runs. A quadratic pattern that must stay under the 100 ms timeout can only sit in a narrow band above the 64 ms bar, so this plant lands on either side of it from run to run. The bar itself did not move.

Test plan

  • Darling.Tests and Lite.Tests build with 0 warnings and 0 errors.
  • PgSettingRedactorTests together with DocCommentHygiene: 216 total, 0 failed.
  • Full Darling.Tests run from this branch: 17425 total, 0 failed, 1142 skipped (live tests that need DARLING_TEST_PG), and the runner reported 1 as not run (it did not say which).
  • Full Lite.Tests run from this branch: 5668 total, 0 failed, 0 skipped.
  • A loaded-machine run of the full suite: the original failure needs one and it was not reproduced here.

The full runs above were made from this branch only. The two sibling test-fix PRs (#4815 and #4817) branch from dev on their own, so those runs do not include their changes.

CHANGELOG

SECTION: None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant