test(dispatch): e2e regression test for 5-stack + Rust dispatch (#550) - #553
Merged
Conversation
PR #537 fixed a critical silent-no-op bug: 5 stacks (crystal, dart, fsharp, kotlin, zig) had no dispatch case in scripts/regret.js. The if-else chain had no else clause, so unmatched stacks exited 0 doing nothing. Existing tests called validate_<stack>.sh directly, bypassing the dispatch layer — the bug was invisible to CI. This test ensures the bug class can never regress. 4 test groups: 1. Static source check (6 tests, always run) — asserts regret.js source contains 'stack === <name>' + references to capture_<name>.sh and validate_<name>.sh for all 6 stacks. 2. Dispatch invocation (12 tests, always run) — builds temp fixture per stack, runs 'regret.js capture|validate', asserts stdout contains the script name (proves dispatch fired). Uses regret.js's own run() console.log as the dispatch signal — works regardless of whether the runtime is installed. 3. Full E2E (12 tests, per-stack skip if runtime missing) — cd into existing proof project, run capture+validate, assert exit 0 + fresh .regret + 'PASS' in output. 4. Negative test (1 test, always run) — uses unsupported stack name to reproduce the exact PR #537 silent-no-op (exit 0, no dispatch line), proving Group 2 would catch a removed dispatch case. Result: 20 tests, all pass. Zero runtimes installed in dev env, so Group 3 skipped — but Groups 1, 2, 4 (the regression catchers) all run without any runtime dependency. Cleanup: no .regret files modified (all E2E skipped). Temp fixtures cleaned via try/finally rmSync. after() hook does git checkout for standard proof projects; rmSync for dart's temp working copy. Closes #550
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR #537 fixed a critical silent-no-op bug: 5 stacks (crystal, dart, fsharp, kotlin, zig) had no dispatch case in
scripts/regret.js. The if-else chain had no else clause, so unmatched stacks exited 0 doing nothing. Existing tests calledvalidate_<stack>.shdirectly, bypassing the dispatch layer — the bug was invisible to CI.This test ensures the bug class can never regress.
Test design — 4 groups, 20 tests total
stack === '<name>'+ references to capture/validate scriptsKey design decision
Dispatch signal = regret.js's own
run()helper which doesconsole.log($ ${cmd} ${cmdArgs.join(' ')})BEFORE spawning child. So stdout containing script name = definitive proof dispatch fired. Works regardless of whether runtime is installed.Verification
node --test tests/dispatch-e2e.test.js→ 20 tests, 20 pass, 0 fail, 0 skippedCleanup
Test plan
Closes #550