LTRAC-2154: Cache routing KV reads at the Cloudflare location for 5 minutes - #3261
Merged
Merged
Conversation
… minutes Workers KV caches reads at each location for 60s by default, the same window as the in-process L1. The L1 therefore re-reads exactly as the location's copy expires, so load testing showed nearly every read going to central storage (~88ms mean, cold and warm alike). Read with a 300s cacheTtl so L1 re-reads hit the location cache instead. Refs LTRAC-2154 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: e2cc228 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Unlighthouse Performance Comparison — VercelComparing PR preview deployment Unlighthouse scores vs production Unlighthouse scores. Summary ScoreAggregate score across all categories as reported by Unlighthouse.
Category Scores
Core Web Vitals
|
…ache A value read just before its expiryTime stays in the location cache for the full cacheTtl, so a storefront status change can take up to ~11 minutes to reach a location instead of ~7. Note it in the adapter and the changeset rather than shortening the status window, which every KV adapter shares. Refs LTRAC-2154 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
parthshahp
approved these changes
Oct 2, 2026
Comment on lines
+32
to
+41
| // How long each Cloudflare location caches a read. The default (60s) matches | ||
| // the in-process L1 window, so the L1 always re-reads just as the location's | ||
| // copy expires and nearly every read goes to central storage. Outlasting the L1 | ||
| // lets those re-reads hit the location cache. Kept at the 5-minute storefront | ||
| // status window, the shortest freshness window `with-routes` stores. | ||
| // | ||
| // Trade-off: a value read just before its `expiryTime` stays cached here for | ||
| // the full TTL, so a storefront status change (maintenance, launch) can take up | ||
| // to ~11 minutes to reach a location instead of ~7. A refresh write is usually, | ||
| // but not guaranteed to be, visible at once in the location that made it. |
Contributor
There was a problem hiding this comment.
Can we shorten this to make sure the comment is providing good value?
Refs LTRAC-2154 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear: LTRAC-2154
What/Why?
CloudflareKvAdapterreadCATALYST_ROUTES_KVwithout acacheTtl, so each Cloudflare location cached a read for Workers KV's 60s default. That's the same window as the in-process L1 in front of it (SHARED_STORE_RECHECK_MS). The L1 re-reads just as the location's copy expires, so in load testing nearly every routing-cache read went to central KV storage, about 88ms, with cold and warm reads looking the same.Reads now pass
cacheTtl: 300. It's longer than the L1 window, so L1 re-reads hit the location cache, and it's capped at the 5-minute storefront-status window, the shortest freshness windowwith-routesstores. Two tests pin both bounds.Trade-off worth a look (also raised by Codex review): a value read just before its
expiryTimestays in the location cache for the full 5 minutes. So in the worst case, a storefront status change (maintenance mode, launch) can take up to about 11 minutes to reach a location instead of about 7 (status window 5 min + location cache + 1 min L1). Route changes get the same 4 extra minutes on top of their 30-minute window. It's usually sooner: the request that sees the expired value triggers a background refresh, and Cloudflare says a write is usually, though not guaranteed to be, visible at once in the location that made it.We chose to accept and document this rather than shorten the status window, because that window is set in
with-routesand shared by every adapter. Shortening it would add Storefront API calls on Vercel and other hosts that don't have this location cache. Keys that don't exist in KV aren't helped; those reads still go to central storage.Testing
Load-tested with k6 from Dallas (DFW) against a Native Hosting sandbox store with temporary
Server-Timinginstrumentation onwith-routes. The instrumentation isn't part of this PR; it's onjorgemoya/kv-lookup-timing. Guest pages only, all 1,385 requests per run returned 200.The steady run's tail is the central refetch each key makes once per 5 minutes; that run also started right after a redeploy, with an empty location cache. In steady traffic about 97% of lookups are answered by the L1, so averaged over every request the KV cost went from about 2.9ms to 1.8ms. The larger gain is on low-traffic pages and fresh isolates.
Migration
See the changeset. In
core/lib/kv/adapters/cloudflare-kv.ts,RoutesKvNamespace.getnow takes{ type: 'json', cacheTtl }instead of'json', andmgetpassescacheTtl: 300.🤖 Generated with Claude Code