Skip to content

test(adaptive): the #592 settled arm checks its reference window was pinned, and reports VOID and re-arms when it was not - #954

Merged
FumingPower3925 merged 3 commits into
mainfrom
test/790-settled592-pinned-premise
Oct 5, 2026
Merged

FumingPower3925 merged 3 commits into
mainfrom
test/790-settled592-pinned-premise

Conversation

@FumingPower3925

Copy link
Copy Markdown
Contributor

TestAdaptiveSettledRouteRetime592/epoll/settled judged its speedup ratio without checking the ratio's premise: that the reference window, taken while /kv was still settled, was pinned. When the kernel's reuseport hash leaves the /ping probe's worker with no /kv conn (about 2^-8 per run), that window is fast. The ratio then comes out at 1.0, and the run read NOT_FIXED even though every #592 state assertion held (CI run 36360645780: ref_ping_med_ms=0.151 ref_queued_frac=0.000).

What changes (test file only: adaptive_settled_retime_linux_test.go)

  1. The premise is checked. refUnpinned790: a reference window counts as pinned only if its /ping median is at least queuedBar589(delay), which is D/2, 150 ms at the default D. Pinned windows measure 0.29 to 2.10 s and unpinned ones about 0.05 to 0.15 ms, so the bar sits three orders of magnitude from both worlds. The check lives in assertSettledRetimed592, after the state cases (promoted_in_bound, promoted_after, async_promoted_conns). Those still decide first, so a adaptive dispatch never re-times a settled route: a store-backed handler that turns slow runs inline on the engine worker forever (#493 item 4, measured) #592 regression is never VOIDed.
  2. VOID and re-arm. A run whose window was not pinned logs RESULT592 ... verdict=VOID attempt=k/3 and a VOID790 line with the reason. Then runStall589 runs again: a new engine and new conns, so new reuseport hashes. Only a non-VOID attempt decides. stubNowNano does not nest, so the clock stub is released between attempts (releaseClockStub592). This is safe because runStall589 has already joined its engine when it returns.
  3. Three VOIDs fail as an apparatus failure, worded apart from NOT_FIXED: apparatus failure, NOT a celeris#592 regression: the rig could not pin a reference window in 3 attempts. At the default placement that is about 2^-24 per run.
  4. The :255 comment is corrected. Placement is the kernel's hash, and the arm now checks it instead of relying on it.

Not done here: the issue's optional item 4 (checking placement before the flip, which would also cover the controls). It needs a test-only view of which loop owns each fd, and that is engine code under the #443 freeze.

Evidence

All runs used Docker golang:1.27 linux/arm64, 4 CPUs and memlock 8 MiB (the CI shape, so the io_uring arms skip), with go test -race -v -run 'TestAdaptiveSettledRouteRetime592/epoll/settled$'. Verdicts are counted from the RESULT592 lines. CELERIS_589_KVCONNS=1 is the issue's forcing: one /kv conn over 2 workers makes the premise fail on about half the runs.

tree forcing runs verdicts reference window
origin/main 2dd32bc (old rig) KVCONNS=1 12 9 FIXED, 3 NOT_FIXED all 3 NOT_FIXED were UNPINNED (ref median about 0.05 ms, ref_queued_frac=0.000, promoted_in_bound=true, speedup 1.0 to 1.2)
this PR KVCONNS=1 12 0 NOT_FIXED. 10 FIXED (8 on attempt 1, 1 on attempt 2, 1 on attempt 3); 2 apparatus failures (3 VOIDs each) 9 VOID attempts, every one UNPINNED; every FIXED attempt pinned
this PR none (default) 2 full-test runs 2 FIXED on attempt 1, 4 CONTROL_OK, io_uring arms SKIP (memlock) pinned

On the forced runs, 9 VOIDs in 19 attempts is about 1/2, and 2 apparatus failures in 12 runs is close to the expected (1/2)^3 = 1/8. That shows the apparatus path works. At the default placement it is about 2^-24.

Control (the defect must still fail). The celeris#592 re-opener is disabled by commenting out r.reopenSettled() in router.go. This is a local edit and is not committed.

control forcing runs verdicts
re-opener disabled none 2 2 NOT_FIXED (promoted_in_bound=false: /kv was not promoted within 8s)
re-opener disabled KVCONNS=1 6 6 NOT_FIXED on attempt 1, never VOID, including the 4 whose window was unpinned: the state check decides before the premise

Fixes #790

… was pinned, and reports VOID and re-arms when it was not (celeris#790)

The ratio compares the post-promotion /ping median with the median taken
while /kv was still settled. When the kernel's reuseport hash gives the
probe's worker no /kv conn (~2^-8 per run), that window is fast and the
ratio is 1.0 although every #592 state assertion held. The premise is now
checked (ref_ping_med >= queuedBar589 = D/2); a miss is VOID, logged on the
RESULT592 line, and the rig is re-armed with a new engine and new conns up
to 3 times. Three misses fail as an apparatus failure, worded apart from
NOT_FIXED. State failures (promoted_in_bound etc.) still decide first.
@FumingPower3925 FumingPower3925 added this to the v1.6.0 milestone Oct 4, 2026
@FumingPower3925 FumingPower3925 added area/ci CI/CD pipeline testing Testing infrastructure and helpers labels Oct 4, 2026
@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration
  • Configuration used: Repository: goceleris/celeris/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5e62e76a-3650-4af3-a434-3ed3a8e69ab6
📥 Commits

Reviewing files that changed from the base of the PR and between ca2111d and 2d4fea8.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: goceleris/celeris/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f7b2b9f5-e4ec-4022-bb26-2a8169eda966
📥 Commits

Reviewing files that changed from the base of the PR and between 2dd32bc and 79b6e41.

📒 Files selected for processing (1)
  • adaptive_settled_retime_linux_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The settled-route test now treats an unpinned reference window as VOID, resets the clock stub between attempts, and retries up to three times. It reports an apparatus failure if all attempts are VOID.

Changes

Settled-route reference validation and retries

Layer / File(s) Summary
Reference validation and VOID retries
adaptive_settled_retime_linux_test.go
The test detects when the reference median is below the queued bar and returns a distinct sentinel. It retries VOID runs up to three times, releasing the clock stub between attempts, then reports an apparatus failure if all attempts are VOID.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other · Severity of issue fixed: Low

Suggested labels: bug

Merge Risk: ⚪ Minimal · up to 79b6e

Unpinned measurements are retried rather than reported as regressions. No merge-blocking issue remains after normal checks.

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Title check ⚠️ Warning The title uses valid Conventional Commit form and describes the test change, but it fixes issue #790 and does not end with that issue reference. Append the issue reference, for example: test(adaptive): the #592 settled arm checks its reference window was pinned, and reports VOID and re-arms when it was not (celeris#790).
✅ Passed checks (3 passed)
Check name Status Explanation
Description check ✅ Passed The description explains the unpinned reference-window problem, the VOID and retry behavior, and the reported validation results.
Linked Issues check ✅ Passed [#790] adaptive_settled_retime_linux_test.go checks the reference median against queuedBar after the promotion and async-connection state checks. An unpinned window returns a distinct VOID error. …
Out of Scope Changes check ✅ Passed The diff changes only adaptive_settled_retime_linux_test.go. The premise check, retry handling, clock-stub release, and placement-comment correction all support [#790]. No unrelated changes are pres…
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Oct 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@FumingPower3925
FumingPower3925 marked this pull request as ready for review October 5, 2026 00:39
@FumingPower3925
FumingPower3925 merged commit 8476e8b into main Oct 5, 2026
17 of 18 checks passed
@FumingPower3925
FumingPower3925 deleted the test/790-settled592-pinned-premise branch October 5, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci CI/CD pipeline testing Testing infrastructure and helpers

Projects

None yet

1 participant