Skip to content

feat(sync): trust cluster -- pull --theirs, manifest<->disk invariant, isDisabled round-trip, never-fetched guard (#466 #467 #472 #497) - #506

Merged
padak merged 1 commit into
mainfrom
feat/sync-trust-cluster-466-467-472-497
Jul 20, 2026
Merged

feat(sync): trust cluster -- pull --theirs, manifest<->disk invariant, isDisabled round-trip, never-fetched guard (#466 #467 #472 #497)#506
padak merged 1 commit into
mainfrom
feat/sync-trust-cluster-466-467-472-497

Conversation

@padak

@padak padak commented Jul 20, 2026

Copy link
Copy Markdown
Member

Summary

Four sync-engine reliability fixes ("sync trust cluster"), shipped together as 0.72.0. They close the failure modes reported by production users maintaining ~20 projects via kbagent sync where production is also edited directly by other people:

Closes #466, closes #467, closes #472, closes #497.

1. sync pull --theirs — supported reconcile path (#466)

The reported workaround for a drifted tree was hand-editing .keboola/manifest.json. --theirs replaces that: remote wins everywhere — overwrites locally-modified configs and rows, restores deleted/missing files, and resolves true merge conflicts by taking the remote version instead of aborting. Works standalone (no --force needed); the SYNC_CONFLICT error message now points at it.

2. Manifest⇔disk invariant (#466 pt3 + #472 pull side)

  • Plain sync pull now re-materializes a tracked config whose local dir was deleted, even when the remote is unchanged. Previously it reported Already up to date and the (CLAUDE.md-documented!) delete-dir-then-pull workaround silently did nothing.
  • Pull can no longer register a manifest entry without writing its files: the remote_unchanged fast path now requires the local _config.yml to exist — the exact guard the row-level branch already had. This kills the name-collision phantom (pull_hash: "", no dir) at the source.

3. Never-fetched guard (#472 push side)

Reproduced on v0.71.0: a phantom manifest entry (empty pull_hash, no dir — left by a pre-0.72 name-collision pull) made sync push --force --dry-run plan DELETE of a remote config nobody ever deleted. Now such entries are excluded from delete planning, reported as never_fetched (JSON key + human warning on diff/push/status), and healed by the next pull. Deleting a properly-pulled config locally still deletes on push — the GitOps delete flow is unchanged (covered by a contrast test + verified live).

4. Config-level isDisabled round-trip (#467)

The components list API returns isDisabled, but the serializer never read it — a flow disabled in production was byte-identical to an enabled one locally, and sync diff reported "in sync". Now:

  • pull writes a sparse is_disabled: true into _config.yml (absent = enabled, so existing trees do not mass-diff after upgrade; hashing treats absent == false at the root only),
  • sync diff surfaces enabled/disabled drift in both directions,
  • push sends the state when the key is present (is_disabled: false re-enables; absent key leaves remote untouched), for configs and rows; config new --push / sync clone create disabled configs when the local file says so.

5. Adopted-by-id manifest writeback (#497)

Pushing an untracked file whose _keboola.config_id resolves on the branch (the #482 adopt-update path) now also writes the manifest entry with fresh hashes, so the next diff reads a stable entry and a later local deletion is detected.

6. UX honesty fixes

Verification

  • Live E2E against project 5946 (us-east4.gcp), driven through the CLI only: phantom push guard (before: planned DELETE; after: warning + no delete), deleted-dir refetch, --theirs conflict resolution (config + row), isDisabled push→pull→drift round-trip verified via config detail, adopted-by-id writeback (manifest entry restored, follow-up diff clean), --all-projects --theirs, and the preserved GitOps delete flow (twins cleaned up via delete-dir + push).
  • 32 new unit tests (tests/test_sync_reconcile.py + additions to config-format/diff-engine/CLI/client tests), 1 new live E2E test (test_sync_theirs_reconcile_and_is_disabled in tests/test_e2e.py, runs under make test-e2e).
  • Full suite: 4400 passed; ruff / ruff format / ty clean; make changelog-check, skill-check, command-sync-check, version-check green.
  • Docs updated per the plugin synchronization map: CLAUDE.md, context.py AGENT_CONTEXT, commands-reference.md, gotchas.md (tagged since v0.72.0), sync-workflow.md (new "Reconciling a drifted tree" section), keboola-expert.md (byte budget respected).

Compatibility notes

  • Existing trees do not mass-diff after upgrade: is_disabled hashing treats absence as false, and only genuinely disabled configs surface (as remote_modified — pull to adopt the field).
  • Behavior change: delete-dir-then-pull now restores the dir (previously no-op). Delete-dir-then-push still deletes the remote config — the direction of the command picks the winner, as documented in the new gotchas entry.

Open in Devin Review

…, isDisabled round-trip, never-fetched guard, adopted-by-id writeback (#466, #467, #472, #497)

Four reliability fixes that make kbagent sync safe against production trees
edited by other people, released together as 0.72.0:

- sync pull --theirs (#466): supported 'discard local, take production'
  reconcile path. Overwrites locally-modified configs and rows, restores
  deleted/missing files, resolves true merge conflicts by taking remote
  instead of aborting. SYNC_CONFLICT error now points at it.
- manifest<->disk invariant (#466/#472): pull re-materializes a tracked
  config whose local dir was deleted (previously silent 'Already up to
  date'), and can no longer register a manifest entry without writing its
  files (the remote_unchanged branch now requires the file to exist,
  matching the row-level guard that already did).
- never-fetched guard (#472): a manifest entry with an empty pull_hash and
  no local files (phantom left by a pre-0.72 name-collision pull) is
  excluded from delete planning -- push --force can no longer DELETE a
  remote config that was never fetched. diff/push/status surface it as
  never_fetched with a warning; the next pull materializes it. Local
  deletion of a properly-pulled config still deletes on push.
- isDisabled round-trip (#467): pull writes sparse 'is_disabled: true'
  into _config.yml (absent = enabled, so existing trees do not mass-diff),
  diff surfaces enabled/disabled drift, push sends the state when the key
  is present (absent leaves remote untouched). Config + row level;
  create paths (config new --push, sync clone) honor it too.
- adopted-by-id writeback (#497): pushing an untracked file whose
  _keboola.config_id resolves on the branch now writes the manifest entry
  with fresh hashes, so follow-up diffs are stable and a later local
  deletion is detected.
- sync status all-clear now says it is a LOCAL-only check and points at
  sync diff for remote drift; row-only pull writes no longer report
  'Already up to date'.

All behaviors live-verified against project 5946 (us-east4.gcp): phantom
push guard, deleted-dir refetch, --theirs conflict resolution (config +
row), isDisabled push/pull/drift, adopted-by-id manifest writeback, and
the preserved GitOps delete flow. 32 new unit tests + 1 new live E2E test
(test_sync_theirs_reconcile_and_is_disabled); full suite 4400 passed.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment