Skip to content

feat: let a consumer scope the mutation cache fingerprint to the tests that reach the diff - #54

Closed
sinemacula-ben wants to merge 1 commit into
masterfrom
feat/scope-the-mutation-cache-fingerprint
Closed

feat: let a consumer scope the mutation cache fingerprint to the tests that reach the diff#54
sinemacula-ben wants to merge 1 commit into
masterfrom
feat/scope-the-mutation-cache-fingerprint

Conversation

@sinemacula-ben

@sinemacula-ben sinemacula-ben commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

The mutation incremental cache is keyed on a hash of every test in the repository, so any test edit discards it. Safe — Stryker cannot invalidate a static mutant when its protecting tests change — but on a repository that edits tests on every push, the cache never warms.

Measured on platform-landing-zone-cdk: every scoped run on a feature branch logged No incremental result file found ... a full mutation testing run will be performed, at 12–18 minutes. Ten pushes to one PR paid it ten times, four of them for a README, a lint suppression, or a test the mutated code cannot reach.

The change

A new optional input, mutation-fingerprint-script, naming an npm script that prints a hash of the tests that can reach the scoped diff.

  • Unset — the default — behaves exactly as today.
  • The lockfile and mutation tooling config stay in the key either way; the consumer only answers which tests matter.
  • A script that fails, or prints nothing, fails the gate rather than falling back to a key nobody can trust.

Scoped job only. The scheduled sweep already runs --force, so its restore is moot and it is untouched.

The risk

A fingerprint that misses a test able to kill a static mutant reuses that verdict, so a PR could go green on a stale pass. Bounded by the scheduled sweep, which reruns from scratch and is blocking. The judgement sits with the consumer because only the repository knows its import graph.

Inert until a consumer sets the input; the platform-landing-zone-cdk side follows separately.

@sinemacula-ben
sinemacula-ben force-pushed the feat/scope-the-mutation-cache-fingerprint branch from f8433c0 to 45a2fa5 Compare September 5, 2026 16:05
…s that reach the diff

The incremental cache is keyed on a hash of every test in the
repository, so any test edit throws it away. That is safe - Stryker
cannot invalidate a static mutant when the tests protecting it change,
so a coarse key is the honest default - but on a repository that edits
tests on every push it means the cache never once warms.

Measured on platform-landing-zone-cdk: every scoped run on a feature
branch reported `No incremental result file found, a full mutation
testing run will be performed`, and cost twelve to eighteen minutes.
Ten pushes to one pull request paid that in full, ten times, including
four that changed only a README, a lint suppression, or a test in a
directory the mutated code cannot reach.

`mutation-fingerprint-script` names an npm script that prints a hash of
the tests that can reach the scoped diff. The lockfile and the mutation
tooling's own configuration stay in the key either way, because a change
to those can move a verdict whichever tests ran. Unset, the behaviour is
exactly as before.

It is the consumer's call because only the repository knows its own
import graph, and it is the consumer's risk: a fingerprint that misses a
test able to kill a static mutant reuses that verdict until the
scheduled sweep, which reruns from scratch and is blocking. A script
that fails, or prints nothing, fails the gate rather than quietly
falling back to a key nobody can trust.
@sinemacula-ben
sinemacula-ben force-pushed the feat/scope-the-mutation-cache-fingerprint branch from 45a2fa5 to c18fd59 Compare September 5, 2026 16:05
@sinemacula-ben

Copy link
Copy Markdown
Contributor Author

Closing: the justification does not hold up. I traced the slow runs to the incremental cache never being written, but the cause was not this fingerprint - four consecutive runs were cancelled mid-flight by the next push, and a cancelled run never reaches the cache-save step. Two consecutive pushes with an identical fingerprint both ran cold, which rules this out as the cause.

The narrower key is still true, but it only ever speeds up a re-run on the same branch, which is not worth an input on a shared workflow.

@sinemacula-ben
sinemacula-ben deleted the feat/scope-the-mutation-cache-fingerprint branch September 5, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant