Skip to content

test(relay): stabilize per-channel replay boundary coverage - #378

Merged
tellaho merged 1 commit into
mainfrom
tho/relay-replay-boundary-test
Sep 28, 2026
Merged

tellaho merged 1 commit into
mainfrom
tho/relay-replay-boundary-test

Conversation

@tellaho

@tellaho tellaho commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Overview

Category: infrastructure

User Impact: No application behavior changes; replay/unread regression coverage keeps its real boundary while avoiding fixture signing inside the behavior test's timeout.

Problem: The quiet-channel replay test intermittently exceeds its 5-second timeout on main because the same test both constructs 502 signed events and verifies 501 socket deliveries. Fixture signing performs its own signature verification, consuming roughly half the local case time before the behavior is exercised.

Solution: Prepare the 501-event hot-channel history in a scoped beforeAll and pin the behavior test's clock. Keep real socket verification, session ingestion, EOSE, quiet-channel unread/attention assertions, and conservative replay evidence; add exact hot-channel delivery assertions without changing timeouts or production code.

Changes

File changes

src/features/relay/live.test.ts
Separate large signed-fixture setup from the replay behavior budget. Preserve 501 distinct hot fixture events competing for a 500-event filter limit, and assert 500 hot unread messages plus the quiet mention survive the real socket/session path without incoming alerts.

Reproduction steps

  1. Run bin/pnpm exec vitest run src/features/relay/live.test.ts; all 59 tests should pass with the original timeout.
  2. Check the replay case: 500 hot messages and one quiet mention are delivered, quiet unread and attention are both 1, no replay alerts fire, and replay remains limited.
  3. To verify regression sensitivity, temporarily replace the initial per-channel singleton filters in live.ts with the former shared multi-channel filter. The quiet-message delivery assertion must fail; restore production code afterward.

Validation and timing

Local macOS arm64, Hermit Node 24.18.0, Vitest 4.1.11; baseline 5ce7836b, final test blob 6bf6cc3a9e943cda315737b8e5f5bbb17c05c5bf (unchanged by commit hooks).

Same full-file command Before After
Quiet-channel case 1,352 ms 668 ms
Sum of individual case durations 2,601 ms 1,908 ms
Full file, including fixture hooks 2,601 ms 2,602 ms

Signing cost is separated, not eliminated: the new fixture hook accounts for about 694 ms. This is timeout-budget isolation, not a suite-speedup claim. A separate CPU profile confirmed signature operations dominate; these comparisons use unprofiled runs.

  • Full Vitest run: BUZZ_TEST_WORKERS=2 bin/pnpm exec vitest run — 4,988 tests / 415 files passed, 212.77 s elapsed, 241.03 s summed file execution; affected case 669 ms. The run used the final working-tree bytes above before commit.
  • Mutation check: the former combined filter fails specifically at delivered.has(quiet.id); production code restored, then the full suite passed.
  • Pre-commit formatting/lint and pre-push TypeScript, related tests (60), design types and guards passed at 01473d51.
  • Existing main failure: affected case 5.963 s; 4,949 other tests passed. That hosted run took 346.12 s elapsed / 607.05 s summed test execution. Local and hosted totals are not comparable benchmarks.

Hosted validation of this change is pending. No local timeout reproduced, no retries/timeouts increased, no browser coverage added or removed. No independent review or human validation claimed; this PR remains draft.

Prepare the large signed replay fixture before the behavior test and pin its clock. Keep real socket signature verification, the 500-event filter boundary, and quiet-channel unread evidence intact; assert the hot-channel delivery count as well.

Co-authored-by: Carl <acda9e433d19dcd0e6b6840f7f4b98f3a56f1fab98049d444c087019e6d36560@buzz.block.builderlab.xyz>
Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
@tellaho
tellaho marked this pull request as ready for review September 28, 2026 22:33
@tellaho
tellaho requested review from a team, comp615 and wesbillman as code owners September 28, 2026 22:33

@baxen baxen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 LGTM. Test-only change: fixture signing moves to a scoped beforeAll, clock pinned so the fixed-timestamp fixtures stay inside the filter's since window, and the per-channel replay assertions are kept plus tightened (501 delivered, hot 500/0). CI green at 01473d5.

@tellaho
tellaho merged commit 23929d7 into main Sep 28, 2026
14 checks passed
@tellaho
tellaho deleted the tho/relay-replay-boundary-test branch September 28, 2026 22:53
cynfria pushed a commit that referenced this pull request Sep 28, 2026
…sh-followup

* origin/main:
  Add message-level read and unread controls (#352)
  feat: add per-category notification alert sounds with app-owned playback (#356)
  test(relay): stabilize per-channel replay boundary coverage (#378)
  fix(design-system): keep button labels single-line and corners capsule-shaped (#357)

Signed-off-by: Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants