Skip to content

Key the file cache on project-relative paths, so a cache survives a change of checkout - #8490

Open
webard wants to merge 1 commit into
rectorphp:mainfrom
webard:feat/relative-cache-keys
Open

Key the file cache on project-relative paths, so a cache survives a change of checkout#8490
webard wants to merge 1 commit into
rectorphp:mainfrom
webard:feat/relative-cache-keys

Conversation

@webard

@webard webard commented Sep 12, 2026

Copy link
Copy Markdown

Closes #8488

ChangedFilesDetector builds its per-file cache key from the file's absolute path, so the whole cache is bound to one location on disk. This keys it on the path relative to the project instead, reusing the existing FilePathHelper::relativePath() - already anchored on getcwd(), and already the form Rector reports to users. hashFile() keeps the absolute path, because it reads the file.

Numbers

A 12 762-file project. A cache is built in one checkout, then copied into a second checkout of the same commit at a different absolute path (a git worktree - the CI-cache-to-laptop case is the same shape):

build the cache run in the second checkout cache entries afterwards
before (absolute keys) 168 s 964 s 24 972
after (relative keys) 260 s 77 s 12 486

Today a copied cache is not merely useless - it is worse than no cache. Every lookup misses, so the second checkout writes a second complete set of entries under its own paths (12 486 → 24 972) while re-analysing everything, and the run takes ~5.7× longer than simply building from scratch would have. With relative keys the second checkout reads the cache it was given, writes nothing new, and finishes in 77 s.

Caveats

  1. Existing caches invalidate once. Old keys are hashes of absolute paths and will not be found; the first run after upgrading re-analyses everything and writes the new keys. No stale entry is ever read, so there is no correctness window - only one slow run.
  2. Files outside the project keep a ../-prefixed relative path. That is still stable, because the anchor does not move relative to them either.
  3. getcwd() as the anchor is an assumption Rector already makes - FilePathHelper is built on it and every reported path depends on it.

Happy to rework the anchor into an explicit dependency if you would rather not lean on getcwd() here.

@TomasVotruba

Copy link
Copy Markdown
Member

Thanks for the feature.

Can we get a CI test case that would ensure this type of run is under x seconds? To avoid regressions in the future

@TomasVotruba

Copy link
Copy Markdown
Member

Why is the build getting slower on 1st run?

@webard

webard commented Sep 12, 2026

Copy link
Copy Markdown
Author

Sorry, I didn't notice. The 260s is measurement noise, not a cost of the change.

Here is the controlled pair instead: same checkout, back to back, isolated, and with relative running first so it carries any ordering disadvantage:

full cold build entries written
absolute keys 132 s 12 486
relative keys 139 s 12 486

And on a single 281-file module, cold then warm: 28.2 s / 16.2 s before, 28.8 s / 16.5 s after.

Mechanically there is nowhere for a real cost to hide: the change adds one makePathRelative() string transform per key computation — no extra I/O, no extra file reads.

On the timed CI test - I would rather not assert wall-clock seconds there, because the threshold would be a property of the runner, not of the code, and it would flake on a busy machine. But the invariant worth protecting is deterministic and easy to pin: a cache built under one directory is reused under another. A test can cache a file in a temp dir, move the project to a second temp dir, chdir() there and assert hasFileChanged() is false (and that no second set of entries appears). That fails loudly the day someone puts an absolute path back into the key, without depending on how fast CI is that day.

Happy to write it in this PR if that shape works for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Cache keys are hashed absolute paths, so the cache can never be reused across checkouts (CI → local, git worktrees)

2 participants