Problem
The tests/test_*.sh directory has grown to ~135 loose shell scripts, each re-implementing ~40 lines of boilerplate (perry-binary + runtime-lib detection, temp dir, compile/run error handling, diff). Most of them are parity tests: they compile a .ts/.js snippet with Perry and diff its stdout against Node. Two structural problems:
- No CI coverage.
run_tests.sh (which globs these) is not invoked by any CI job. The cargo-test job only runs Rust #[test]s; the parity job (which does run .ts corpus tests) is tag-gated. So these shell tests only ever run when someone runs run_tests.sh locally.
- Disorganized & duplicative. Bespoke per-case filenames, no shared harness, no domain grouping.
Meanwhile a mature parity system already exists: drop a .ts into test-files/ (or test-parity/node-suite/<module>/) and the harness auto-discovers it, compiles with Perry, runs under Node, and diffs. Intentional Perry-vs-Node divergences use a stored test-parity/expected/<name>.txt; non-zero exits use expected-exit/.
Proposal
Migrate the parity-shaped shell tests into the parity corpus, so the .ts is the test:
- Pure "match Node" cases →
test-files/test_parity_<case>.ts (auto node-diff, no boilerplate).
- Node-module-specific cases →
test-parity/node-suite/<module>/<case>.ts (grouped by domain; supports subdirs + the multi/ convention for multi-module fixtures).
- Intentional divergences (e.g.
new Function throwing, dynamic-import diagnostics) → add test-parity/expected/<name>.txt.
- Delete the migrated shell wrappers.
Caveats / not purely mechanical
- stderr assertions (e.g.
test_dynamic_function_probe_fallback_stderr): the parity harness diffs stdout; these need adapting (move the assertion to stdout, or keep as a shell test).
- Multi-module tests →
node-suite/<x>/ with a multi/ deps folder.
- A handful genuinely can't be pure parity and stay as shell tests.
Interim
PR #5874 added tests/_perry_test_lib.sh (shared harness) and adopted it in two tests, which reduces the boilerplate for any shell tests that remain after migration.
Scope
Start with the ~40 Zod-compat regression tests added in #5874; then sweep the remaining ~95. Best done as its own PR (test-convention change, independent of Zod).
Problem
The
tests/test_*.shdirectory has grown to ~135 loose shell scripts, each re-implementing ~40 lines of boilerplate (perry-binary + runtime-lib detection, temp dir, compile/run error handling, diff). Most of them are parity tests: they compile a.ts/.jssnippet with Perry and diff its stdout against Node. Two structural problems:run_tests.sh(which globs these) is not invoked by any CI job. Thecargo-testjob only runs Rust#[test]s; theparityjob (which does run.tscorpus tests) is tag-gated. So these shell tests only ever run when someone runsrun_tests.shlocally.Meanwhile a mature parity system already exists: drop a
.tsintotest-files/(ortest-parity/node-suite/<module>/) and the harness auto-discovers it, compiles with Perry, runs under Node, and diffs. Intentional Perry-vs-Node divergences use a storedtest-parity/expected/<name>.txt; non-zero exits useexpected-exit/.Proposal
Migrate the parity-shaped shell tests into the parity corpus, so the
.tsis the test:test-files/test_parity_<case>.ts(auto node-diff, no boilerplate).test-parity/node-suite/<module>/<case>.ts(grouped by domain; supports subdirs + themulti/convention for multi-module fixtures).new Functionthrowing, dynamic-import diagnostics) → addtest-parity/expected/<name>.txt.Caveats / not purely mechanical
test_dynamic_function_probe_fallback_stderr): the parity harness diffs stdout; these need adapting (move the assertion to stdout, or keep as a shell test).node-suite/<x>/with amulti/deps folder.Interim
PR #5874 added
tests/_perry_test_lib.sh(shared harness) and adopted it in two tests, which reduces the boilerplate for any shell tests that remain after migration.Scope
Start with the ~40 Zod-compat regression tests added in #5874; then sweep the remaining ~95. Best done as its own PR (test-convention change, independent of Zod).