Changelog CDN fetchers: skip unchanged folders via shallow registry maps

[cotti]

Implements the consumer half of the shallow per-tree registry maps introduced in #3738 (stacked on that PR's branch): the bundle CDN fetcher now consults bundle/registry.json before fetching per-product registries, and skips folders whose content demonstrably hasn't changed.

Refs: elastic/docs-eng-team#737, elastic/docs-eng-team#688, #3738.

What it does

CdnChangelogFetcher fetches the tree's shallow map (bundle/registry.json, shape {"<product>": "<token>"}) once per fetcher run, memoized per base URI. For each product:

• Token match + warm cache → the per-product bundle/{product}/registry.json fetch is skipped and the locally cached registry is reused. Bundle content then resolves through the existing ETag-keyed cache, so an unchanged folder with a warm cache produces zero per-folder requests (asserted in tests).
• Token mismatch, product missing from the map, or no cached state → the flow is byte-identical to today's: fetch the registry, download bundles, and record the fresh registry under the folder's current token for the next run.

Graceful degradation (non-negotiable, preserved)

A shallow map that is absent (404), unparseable, or fails to fetch degrades to null inside FetchShallowMapAsync — a single debug log, no errors, no warnings — and every per-product registry is fetched exactly as before the map existed. Pre-cutover CDNs and buckets without maps keep working unchanged. Tokens are treated as fully opaque: compared with ordinal string equality only, never parsed.

Cache/token bookkeeping design

• The registry text is cached under key registry-{product}-{token} in the fetcher's existing memory + disk cache ({ApplicationData}/changelog-bundles/), the same store and conventions the ETag-keyed bundle cache (changelog-{product}-{file}-{etag}) already uses. Embedding the token in the key makes a token mismatch a plain cache miss under the new key — no separate "last-seen token" state to keep consistent.
• This keying is sound because the producer (ShallowRegistryReconciler) excludes group manifests from the token digest precisely so rewriting a manifest cannot invalidate consumer caches: same token ⇒ same folder listing ⇒ same derived registry.
• The shallow map itself is deliberately never cached to disk — it is the freshness signal — and is fetched at most once per fetcher run regardless of product count.
• Defense-in-depth: a token that is not a safe single path segment is ignored (treated as "no token") before it can be joined into a cache file path, mirroring the fetchers' existing file-name validation.

Deliberately not covered: CdnChangelogEntryFetcher

The changelog-pool entry fetcher (changelog/{org}/{repo}/{branch}) has no local cache of any kind — it downloads entries and returns them directly to the bundle command. A matching pool token could therefore never skip anything: there is no cached state a skip could reuse, and honoring the issue's "no behavior change" constraint would require inventing a parallel content store for entry YAML. Per the review direction ("scope the opt-out to whatever cache genuinely exists; correctness over cleverness"), that fetcher is left unchanged; fetching changelog/registry.json there would only add a wasted request per run. If a pool-level entry cache is ever introduced, the same token-keyed pattern applies directly.

Test plan

All in CdnChangelogFetcherTests (fake HttpMessageHandler + MockFileSystem, per the existing pattern):

• Map absent (404) → identical behavior to today (registry + bundles fetched, no diagnostics).
• Map unparseable → identical behavior, no crash.
• Token match with warm cache → the fake handler sees only the single map request; results served entirely from cache.
• Token mismatch → normal fetch, fresh registry recorded under the new token.
• Token present but cold cache → normal fetch on run one, full skip on a second run sharing the disk cache.
• Partial run across two products (one matching, one not) → only the changed product's folder is hit; the map is fetched exactly once.
• Existing ETag-cache tests updated for the one extra map probe per run.

Checks run: dotnet format (clean), ./build.sh build --skip-dirty-check (pass), dotnet publish src/tooling/docs-builder -c Release AOT publish (zero trim/AOT warnings; the shallow map type is registered on the source-generated ChangelogRegistryJsonContext), dotnet test for Elastic.Documentation.Configuration.Tests (608 passed) and Elastic.Changelog.Tests (865 passed). No CLI surface changes, so no docs/cli-schema.json regen.

[github-actions]

Requesting changes: the shallow-map cancellation/timeout path currently breaks graceful degradation and can fail all product fetches for a base URI after a single map timeout.

---

What is this? | From workflow: PR Review

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

FetchShallowMapAsync currently treats every OperationCanceledException as fatal, but HttpClient timeout is surfaced as TaskCanceledException (which inherits OperationCanceledException) even when the caller token was not canceled. In that case the map fetch does not degrade to null and instead faults the shared Lazy<Task<...>> in _shallowMaps, so all subsequent product fetches for the same base URI fail immediately in this run.

That creates a real behavior regression versus the intended "map failures degrade to normal per-product registry fetch" path.

Please distinguish caller cancellation from transport timeout/transient cancellation here (for example, rethrow only when ctx.IsCancellationRequested, otherwise log+return null), and avoid keeping a failed/canceled lazy entry cached for the rest of the run.

Stale: this finding (shallow-map cancellation/timeout swallowing) was fixed in 97fef92 (pushed 2026-08-26T13:43:04Z, after this review was submitted at 13:01:51Z), verified by new regression tests.

[Mpdreamz]

Reviewed. Fixed a real bug flagged by the automated PR review: the shallow changelog-map fetch treated HttpClient's own request-timeout TaskCanceledException as a genuine cancellation, permanently faulting the shared per-base-URI Lazy<Task<...>> and taking every subsequent product fetch down with it for the rest of the run. Now only rethrows when the caller's token was actually signaled (ctx.IsCancellationRequested); a transport timeout degrades to null like any other transport failure, and a genuine cancellation evicts the faulted cache entry so a later call with a live token can retry. Added three regression tests (FetchAsync_ShallowMapTimesOut_DegradesInsteadOfFaultingLaterFetches, FetchAsync_ShallowMapTimesOut_DoesNotPoisonLaterProductsInTheSameRun, FetchAsync_CallerCancels_PropagatesCancellationRatherThanDegrading). Full test suite passes; CLI schema unchanged.