Key the file cache on project-relative paths, so a cache survives a change of checkout - #8490
Key the file cache on project-relative paths, so a cache survives a change of checkout#8490webard wants to merge 1 commit into
Conversation
|
Thanks for the feature. Can we get a CI test case that would ensure this type of run is under x seconds? To avoid regressions in the future |
|
Why is the build getting slower on 1st run? |
|
Sorry, I didn't notice. The 260s is measurement noise, not a cost of the change. Here is the controlled pair instead: same checkout, back to back, isolated, and with relative running first so it carries any ordering disadvantage:
And on a single 281-file module, cold then warm: 28.2 s / 16.2 s before, 28.8 s / 16.5 s after. Mechanically there is nowhere for a real cost to hide: the change adds one On the timed CI test - I would rather not assert wall-clock seconds there, because the threshold would be a property of the runner, not of the code, and it would flake on a busy machine. But the invariant worth protecting is deterministic and easy to pin: a cache built under one directory is reused under another. A test can cache a file in a temp dir, move the project to a second temp dir, Happy to write it in this PR if that shape works for you. |
Closes #8488
ChangedFilesDetectorbuilds its per-file cache key from the file's absolute path, so the whole cache is bound to one location on disk. This keys it on the path relative to the project instead, reusing the existingFilePathHelper::relativePath()- already anchored ongetcwd(), and already the form Rector reports to users.hashFile()keeps the absolute path, because it reads the file.Numbers
A 12 762-file project. A cache is built in one checkout, then copied into a second checkout of the same commit at a different absolute path (a git worktree - the CI-cache-to-laptop case is the same shape):
Today a copied cache is not merely useless - it is worse than no cache. Every lookup misses, so the second checkout writes a second complete set of entries under its own paths (12 486 → 24 972) while re-analysing everything, and the run takes ~5.7× longer than simply building from scratch would have. With relative keys the second checkout reads the cache it was given, writes nothing new, and finishes in 77 s.
Caveats
../-prefixed relative path. That is still stable, because the anchor does not move relative to them either.getcwd()as the anchor is an assumption Rector already makes -FilePathHelperis built on it and every reported path depends on it.Happy to rework the anchor into an explicit dependency if you would rather not lean on
getcwd()here.