feat: let a consumer scope the mutation cache fingerprint to the tests that reach the diff - #54
Closed
sinemacula-ben wants to merge 1 commit into
Closed
Conversation
sinemacula-ben
force-pushed
the
feat/scope-the-mutation-cache-fingerprint
branch
from
September 5, 2026 16:05
f8433c0 to
45a2fa5
Compare
…s that reach the diff The incremental cache is keyed on a hash of every test in the repository, so any test edit throws it away. That is safe - Stryker cannot invalidate a static mutant when the tests protecting it change, so a coarse key is the honest default - but on a repository that edits tests on every push it means the cache never once warms. Measured on platform-landing-zone-cdk: every scoped run on a feature branch reported `No incremental result file found, a full mutation testing run will be performed`, and cost twelve to eighteen minutes. Ten pushes to one pull request paid that in full, ten times, including four that changed only a README, a lint suppression, or a test in a directory the mutated code cannot reach. `mutation-fingerprint-script` names an npm script that prints a hash of the tests that can reach the scoped diff. The lockfile and the mutation tooling's own configuration stay in the key either way, because a change to those can move a verdict whichever tests ran. Unset, the behaviour is exactly as before. It is the consumer's call because only the repository knows its own import graph, and it is the consumer's risk: a fingerprint that misses a test able to kill a static mutant reuses that verdict until the scheduled sweep, which reruns from scratch and is blocking. A script that fails, or prints nothing, fails the gate rather than quietly falling back to a key nobody can trust.
sinemacula-ben
force-pushed
the
feat/scope-the-mutation-cache-fingerprint
branch
from
September 5, 2026 16:05
45a2fa5 to
c18fd59
Compare
Contributor
Author
|
Closing: the justification does not hold up. I traced the slow runs to the incremental cache never being written, but the cause was not this fingerprint - four consecutive runs were cancelled mid-flight by the next push, and a cancelled run never reaches the cache-save step. Two consecutive pushes with an identical fingerprint both ran cold, which rules this out as the cause. The narrower key is still true, but it only ever speeds up a re-run on the same branch, which is not worth an input on a shared workflow. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The mutation incremental cache is keyed on a hash of every test in the repository, so any test edit discards it. Safe — Stryker cannot invalidate a static mutant when its protecting tests change — but on a repository that edits tests on every push, the cache never warms.
Measured on
platform-landing-zone-cdk: every scoped run on a feature branch loggedNo incremental result file found ... a full mutation testing run will be performed, at 12–18 minutes. Ten pushes to one PR paid it ten times, four of them for a README, a lint suppression, or a test the mutated code cannot reach.The change
A new optional input,
mutation-fingerprint-script, naming an npm script that prints a hash of the tests that can reach the scoped diff.Scoped job only. The scheduled sweep already runs
--force, so its restore is moot and it is untouched.The risk
A fingerprint that misses a test able to kill a static mutant reuses that verdict, so a PR could go green on a stale pass. Bounded by the scheduled sweep, which reruns from scratch and is blocking. The judgement sits with the consumer because only the repository knows its import graph.
Inert until a consumer sets the input; the
platform-landing-zone-cdkside follows separately.