Skip to content

ci: run the raku grammar tests - #6175

Open
prql-bot wants to merge 3 commits into
mainfrom
ci/test-raku-grammar
Open

ci: run the raku grammar tests#6175
prql-bot wants to merge 3 commits into
mainfrom
ci/test-raku-grammar

Conversation

@prql-bot

@prql-bot prql-bot commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

grammars/raku/ ships a full test suite — 81 assertions across 10 t/*.rakutest files — that CI has never run. The test-grammars job triggers on grammars/** (so it already fires when the Raku grammar changes) but only builds and tests the lezer grammar. The one other Raku reference in the config, RAKU_RAKU_ARGUMENTS in .mega-linter.yaml, only compile-checks the sources; it doesn't execute the tests. So the grammar can drift from the language with no signal.

This adds a Raku/setup-raku@v1 step and runs the suite with prove6 inside the existing test-grammars job, rather than a new job, so the check-ok-to-merge gate list doesn't need to change. The two existing lezer steps are renamed to say lezer, since the job now covers two grammars.

There's no regression test here because this is the test coverage — it wires up a suite that already exists.

Verification

I ran the suite locally against Rakudo 2026.07 (the version setup-raku@v1 installs by default):

$ prove6 --lib t/
t/arithmetics.rakutest ... ok
t/arrays.rakutest ........ ok
t/datetime.rakutest ...... ok
t/full_queries.rakutest .. ok
t/identifiers.rakutest ... ok
t/misc.rakutest .......... ok
t/numbers.rakutest ....... ok
t/operators.rakutest ..... ok
t/strings.rakutest ....... ok
t/tuples.rakutest ........ ok
All tests successful.
Files=10, Tests=81,  4 wallclock secs
Result: PASS

The suite is a meaningful gate rather than one that always passes — dropping in a test that asserts PRQL.parse succeeds on garbage input gives Result: FAILED and exit status 1, so the step fails.

zef ships inside the rakudo archive setup-raku installs (share/perl6/site/bin/zef, a directory the action adds to PATH), so zef install App::Prove6 needs no extra setup. It took 39s locally, against ~4s for the tests themselves.

prettier@3.9.6 and actionlint@1.7.12 (the versions pinned in .pre-commit-config.yaml) both pass on the edited workflow.

@vanillajonathan

Copy link
Copy Markdown
Collaborator

Note: If prove6 is installed (using zef) then it is possible to run al tests with one command:

zef install App::Prove6
prove6 --lib t/

@prql-bot

Copy link
Copy Markdown
Collaborator Author

Good call — and it's what grammars/raku/README.md already documents, so the workflow should match it. Switched in e8ce7d1.

zef turns out to already ship inside the rakudo archive setup-raku@v1 installs, at share/perl6/site/bin/zef — and the action adds that directory to PATH — so zef install App::Prove6 works with no extra setup step. It costs 39s against ~4s for the tests themselves, which seems worth it: prove6 reports every failing file, whereas the bash -e loop stopped at the first one.

Verified locally against Rakudo 2026.07
$ prove6 --lib t/
t/arithmetics.rakutest ... ok
t/arrays.rakutest ........ ok
t/datetime.rakutest ...... ok
t/full_queries.rakutest .. ok
t/identifiers.rakutest ... ok
t/misc.rakutest .......... ok
t/numbers.rakutest ....... ok
t/operators.rakutest ..... ok
t/strings.rakutest ....... ok
t/tuples.rakutest ........ ok
All tests successful.
Files=10, Tests=81,  4 wallclock secs
Result: PASS

And that it's a real gate — a probe test asserting PRQL.parse succeeds on garbage input gives Result: FAILED with exit status 1, so the step fails.

@prql-bot prql-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wiring checks out — main in the rules filter covers .github/**, so test-grammars ran on this PR rather than being skipped, and the three new steps all passed on e8ce7d1 (Setup Raku 4s, install 36s, tests 4s). One suggestion inline on the install step.

Comment thread .github/workflows/tests.yaml Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants