Skip to content

Add changelog registry state discovery and reconciliation commands#3717

Open
cotti wants to merge 1 commit into
mainfrom
feat/changelog-registry-reconciliation
Open

Add changelog registry state discovery and reconciliation commands#3717
cotti wants to merge 1 commit into
mainfrom
feat/changelog-registry-reconciliation

Conversation

@cotti

@cotti cotti commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

registry.json manifests (per scope: bundle/{product}/ and changelog/{org}/{repo}/{branch}/) merge additively under optimistic concurrency and are not authoritative for removals or discovery, so they can silently drift from the objects actually in the bucket. Nothing could detect or repair that drift, and backfill planning needs a trustworthy current-state view. This PR adds a changelog registry command group backed by a new Elastic.Changelog/Reconciliation/ service layer:

  • changelog registry inspect (read-only) — compares a scope's private registry against the actual private-bucket objects, prints human-readable diagnostics, and optionally writes a machine-readable RegistryStateSnapshot (--out) that the backfill planner (elastic/docs-eng-team#676) will consume.
  • changelog registry repair (explicit, never implicit; idempotent) — converges the private registry to the actual objects.
  • changelog registry verify-public (strictly read-only) — waits under a bounded retry policy (--max-attempts × --poll-interval-seconds) for scrubber propagation and diagnoses public-side divergence.
  • changelog registry republish (explicit) — re-emits the private ObjectCreated event for selected objects to recover a lost/DLQ'd scrub event.

Divergence taxonomy

Class Meaning
missing Object exists in the scope, registry has no entry for it.
stale Registry entry whose object no longer exists.
corrupt Manifest unparseable, or contains unsafe/duplicate entries.
object-divergent Entry metadata (ETag or target) disagrees with the actual object — including legacy amends whose registry target is null while the parent bundle resolves one.

A manifest with a newer schema_version is reported as UnsupportedSchema (not corrupt): its entries can't be judged by an older tool, and repair refuses to rewrite it because that would silently downgrade the schema. For bundle scopes the expected target is re-derived from the bundle YAML in S3 (including the parent-bundle fallback for legacy amends, mirroring RegistryBuilder); changelog scopes enumerate files only.

Repair semantics

  • Rebuilds the manifest from actual objects through the same conditional-PUT optimistic concurrency as the live upload path (If-Match on update, If-None-Match: * on create). A 412 triggers a full re-inspection — fresh registry read and fresh object listing — before the bounded retry, and the registry is always read before listing, so a concurrent upload either appears in the re-listing or invalidates the precondition; it can never be dropped.
  • Idempotent: a clean scope writes nothing; running repair twice yields no further changes (covered by tests).
  • Audited: every added/removed/corrected entry is logged with before/after values; --dry-run prints the plan without writing; an empty result requires --allow-empty.
  • Marked [RequiresAuth] + [CommandIntent(Intent.Idempotent)] + [MutationScope(MutationScope.Global)] (same for republish); inspect/verify-public are [RequiresAuth] only.

Public-boundary design

The public bucket is scrubber-owned and is a hard write boundary enforced structurally, not by convention: all inspection and public-side comparison code operates exclusively on a read-only reader interface (IS3ScopeReader: list + get only), so no code path in the checker can mutate any bucket. Tests additionally assert zero PutObject/CopyObject/DeleteObject calls on the public-side fake.

Recovery from a lost scrub event is the explicit private-side republish: a metadata-preserving S3 self-copy (CopyObject onto the same key with MetadataDirective: REPLACE, re-supplying the original content type and user metadata) that leaves content and ETag untouched while emitting the ObjectCreated:Copy notification the scrubber Lambda reacts to (src/infra/docs-lambda-changelog-scrubber/Program.cs matches ObjectCreated). The scrubber remains the sole public-side writer.

Docs

  • New supplemental CLI pages under docs/cli/changelog/registry/ (namespace + 4 commands); docs/cli-schema.json regenerated (verified: diff empty ignoring version).
  • docs/development/changelog-bundle-registry.md gains a "State discovery and reconciliation" section, and the registry.json example is corrected to the actual serialized key (e_tag, not etag).

Test plan

45 new tests in tests/Elastic.Changelog.Tests/Reconciliation/ using a stateful in-memory S3 fake (FakeItEasy over IAmazonS3, MD5 ETags matching S3EtagCalculator, real conditional-PUT 412 semantics):

  • Each divergence class detected on seeded state: missing, stale, corrupt (unparseable + unsafe entries), object-divergent (ETag and legacy-amend target)
  • Clean scope reports clean; missing manifest over an empty scope is clean; missing manifest over a populated scope reports per-object missing
  • Repair convergence + idempotence (second run writes nothing); create path uses If-None-Match: *, update path If-Match
  • Repair safety under optimistic-concurrency conflict: concurrent registry update between read and write → 412 → re-inspect → concurrent entry survives
  • Repair rails: dry-run writes nothing; empty scope refused without --allow-empty; newer schema refused
  • Public check never writes (asserted on the fake); bounded wait stops exactly at --max-attempts; converges early when the scrubber catches up mid-poll
  • Republish self-copies only within the private bucket, preserves content/content-type/user metadata, validates file names, --all includes registry.json, partial failures reported
  • Full suite: Elastic.Changelog.Tests 849/849 passed; ./build.sh unit-test green
  • dotnet format + ./build.sh lint clean; ./build.sh build --skip-dirty-check succeeds; dotnet publish -c Release (Native AOT) succeeds with zero trim/AOT warnings; repo docs build validates the new supplemental pages

Closes elastic/docs-eng-team#670

Registries merge additively under optimistic concurrency and are not
authoritative for removals or discovery, so they can drift from the
objects actually in the bucket and nothing could detect or repair that.
Backfill planning needs a trustworthy current-state snapshot, so this
adds a `changelog registry` command group:

- `inspect` (read-only): classifies divergence between a scope's
  private registry.json and the actual private objects — missing,
  stale, corrupt, and object-divergent — and can emit a
  machine-readable RegistryStateSnapshot for the backfill planner.
- `repair` (explicit, idempotent): converges the private registry to
  the actual objects through the same conditional-PUT optimistic
  concurrency as the live upload path, re-inspecting on 412 so
  concurrent uploads are never clobbered. Guarded against empty
  results (--allow-empty) and newer-schema downgrades; fully audited.
- `verify-public` (strictly read-only): waits under a bounded retry
  policy for scrubber propagation and diagnoses public divergence.
  The public bucket is a hard write boundary enforced structurally:
  the comparison operates on a reader interface with no write surface.
- `republish` (explicit): re-emits the private ObjectCreated event via
  a metadata-preserving S3 self-copy to recover a lost or DLQ'd scrub
  event; the scrubber remains the sole public-side writer.

Part of elastic/docs-eng-team#670
@github-actions

Copy link
Copy Markdown

Label error. Requires exactly 1 of: automation, breaking, bug, changelog:skip, chore, ci, dependencies, documentation, enhancement, feature, fix, redesign. Found:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant