Skip to content

create-vale-rule: the recommended check <rule>/.tests/fail --json can never return a finding #370

Description

@thecodedrift

From the 2026-09-21 dogfood, CLI 0.11.2, Vale 3.21.0, recipe topic v8.

The recipe's test section says that if you want to see the raw findings, message text and line numbers, run:

npx @taskless/cli@latest check .taskless/rules/vale/<id>/.tests/fail --json

and "read results there." On 0.11.2 that command returns results: [] for every rule, including one whose test reports the fail fixture firing:

$ npx @taskless/cli test .taskless/rules/vale/mirrored-negation --json
  ok: true (fail fixture fired, pass fixture quiet)
$ npx @taskless/cli check .taskless/rules/vale/mirrored-negation/.tests/fail --json
  results: []

Cause is the one #363 describes: check excludes .taskless/ unconditionally before Vale runs (packages/cli/src/rules/vale/formats.ts). So the recipe recommends a command that its own engine has already emptied. An author who follows it concludes the rule is broken and starts editing a pattern that works. The recipe's own "when a fail document does not fire, work down this list" section makes that worse, because the first suspect it names is the .vale.ini.

The practical consequence for us: the only way to read a rendered %s message was to run check against a real corpus file. That is how we caught a substitution message with its two %s slots in the wrong order, which test reports as ok.

What to do

Either of these fixes it. The second is more useful.

  • Recipe: drop the check <bucket> suggestion, or say it only works when the bucket sits outside .taskless/.
  • CLI: let test print the findings it saw, or give check a way to lint a fixture path on purpose (--include-fixtures, or treat an explicit path under .taskless/**/.tests/ as opted in). Then test --verbose shows the rendered message and the author can read it without a corpus file.

Related: #363.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIRelated to the taskless CLIdocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions