fix(sync): scope diff/push to one branch tree and report orphans (#649) - #654
Merged
Conversation
A production `sync diff` / `sync push` run after `sync pull --branch <dev>` classified every config in the orphaned `main/` tree as `added` with an empty `config_id`, and every dev-only config as `added` with an id -- one push away from duplicating an entire production project. No scaffolding was needed; the documented multi-branch pull was enough. Root cause, two independent legs: 1. The adopt-by-id guard (#482) refused adoption whenever the id was claimed by ANY manifest entry. `tracked_keys` has no branch dimension, so after a pull re-targeted the manifest to the dev branch, every id in `main/` looked like a fork-by-copy and kept the create. 2. `diff()` built the local side from ALL `manifest.configurations`, so entries tracked on the dev branch were compared against the production remote. Both are fixed in the diff computation, which push consumes (so `push` and `push --dry-run` inherit them): - The local side is now scoped to the ONE tree the diff reads -- the source branch path. Entries whose branch resolves to a different tree are excluded and reported under a new additive `orphaned` bucket (`summary.orphaned` + per-entry details and hints; human mode previews the first 10). This also removes a latent inconsistency: `push_update` already reads every file through the source path, so a cross-tree entry was diffed from one file and pushed from another. - The id-claim check is branch-aware. A same-tree claim keeps today's fork-by-copy CREATE (#482/#497 contract intact). A claim only from another tree means the manifest was re-targeted: the file is adopted when its id still resolves on the target branch, and reported as `stale_branch_tree` when it does not. - Branch identity is normalized through the branch tree path, so production spelled `None`, `0` (git-branching pull) or the default branch's numeric id all name the same tree. The branch-scoping logic lives in a new `sync/branch_scope.py` (the untracked walkers moved with it), alongside the `sync/branch_registry.py` extraction from #644; `sync_service.py` is grandfathered at its size and may only shrink. Tests: 13 new (11 service-level covering the issue's repro, the KFR-07 promote flow, the dev-branch control, legacy `branchId: 0`, the normalizer, and the #644 scaffold-adoption interaction; 2 CLI covering the human warning and its preview cap). Full suite green. Docs: sync-workflow.md, branch-workflow.md, gotchas.md, commands-reference.md, AGENT_CONTEXT.
padak
force-pushed
the
fix/649-branch-scoped-diff-orphans
branch
from
August 22, 2026 22:46
bd043ed to
7cde8aa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
After
sync pull --branch <dev>re-targetsmanifest.configurationsto the dev branch, the defaultmain/tree is orphaned on disk. A subsequent productionsync diff/sync pushthen classified every config inmain/asaddedwith an emptyconfig_id, and every dev-only config asaddedwith an id — onesync pushaway from duplicating an entire production project. Verified live on project 5946 (v0.88.0):added: 19where the correct answer is0.No scaffolding or hand-edited manifest is needed to hit it; the documented multi-branch pull is enough. This is the same failure class as #482, on the axis #482's fix does not cover: #482 scoped the untracked-file scan to the source branch subtree, which protects a dev-branch push from orphaned trees. A production push reads
main/as its legitimate source — andmain/is exactly the treepull --branchjust orphaned.Root cause — two independent legs
untracked_key not in tracked_keys).tracked_keysis keyed"{component_id}/{config_id}"with no branch dimension, so after the re-target every id inmain/looked like a fork-by-copy and the create was kept.diff()built the local side from allmanifest.configurationsregardless of the target branch, so configs tracked on the dev branch were compared against the production remote.The fix
Both legs are fixed inside the diff computation, which
pushconsumes — sosync pushandsync push --dry-runinherit the classification unchanged.main/when the target has none — the KFR-07 promote path). Manifest entries whose branch resolves to a different tree are excluded from the changeset and reported under a neworphanedbucket. This also closes a latent inconsistency:_sync_push_ops.push_updatealready reads every file through the source path, so a cross-tree entry was diffed from one file and pushed from another (or failed withFileNotFoundError).unchanged/modified), and reported asstale_branch_treewhen it does not — never re-created.None(CLI),0(branchId=branch_id or 0from a git-branching pull) and the default branch's numeric id (plain pull). All three name the same tree.orphanedis additive (--json): asummary.orphanedcounter plus a details list carryingcomponent_id,config_id,path,branch_id,branch_path,exists_on_target,reason(tracked_on_other_branch/stale_branch_tree) and ahint. Human mode prints a warning naming the fix (sync pullto re-target,branch mergeto promote a dev-only config) and previews the first 10 rows — a re-targeted manifest orphans a whole project at once.Acceptance from the issue holds: the production diff after a dev pull reports
added: 0, the pre-existing configs asunchanged, plus warnings;sync push --dry-runplans no creates.One deviation from the issue's sketch
The issue proposed gating a cross-branch tracked entry on "does its id resolve on the target remote". Implemented as written, that entry and the adopted
main/file for the same config would both enterlocal_configs— the config would be counted twice, with contradictory classifications. The code also proves the entry cannot be pushed correctly anyway (push_updatereads the source tree, not the entry's own tree). So the participation rule is the source tree instead, which is strictly narrower: the KFR-07 "promote the default tree to a dev branch" flow never trips it (there the entries' tree is the source tree, so promote-CREATE still works — covered by a new test), and remote resolution is kept as the signal that words the orphan hint and drives adoption.Structure
The branch-scoping logic lives in a new
src/keboola_agent_cli/sync/branch_scope.py; the two untracked-file walkers moved there with it andsync_service.pynow delegates. It sits alongside thesync/branch_registry.pyextraction that landed with #644 — the two modules are disjoint concerns (which tree a diff reads vs. registering the directory a scaffold is written to) and both delegates coexist insync_service.py. That file is grandfathered at 1655 code lines inscripts/file_size_baseline.jsonand may only shrink, so the extraction is required, not cosmetic: main sits at 1653, this branch brings it to 1604.Tests
13 new, TDD (they reproduce
added == 2against the pre-fix code):TestIssue649ProductionDiffAfterBranchPull(service level,tests/test_sync_service.py): the issue's repro for diff / push /push --dry-run, theorphanedbucket's contents and hints, a stale-tree file whose id no longer resolves, the dev-branch control (unchanged behaviour), the KFR-07 promote-still-creates guard, a healthy single-branch tree (bucket stays empty), a legacybranchId: 0manifest, a unit test for the branch normalizer, and the interaction with config new --output-dir --push writes a scaffold without the created config ID → duplicates on next sync push #644 (aconfig new --push --output-dirscaffold placed in a dev-branch subtree is still adopted by id, never re-created).TestSyncOrphanWarningCli(tests/test_sync_cli.py): human mode warns instead of printing "No differences found", and caps the list.Existing #482 / #497 / #472 regression tests stay green. Rebased onto main after #650/#644 landed; full suite: 6006 passed, 61 skipped.
make lint format-check typecheck skill-check version-check version-gate-check command-sync-check changelog-check check-error-codes check-sentinel-guards loc-checkall green (the onetydiagnostic is the pre-existinghatch_build.pyimport).No version bump and no
changelog.pyentry (per the release-PR convention); the new gotcha is tagged(since vNEXT).Docs synced:
sync-workflow.md,branch-workflow.md,gotchas.md,commands-reference.md,AGENT_CONTEXT(context.py).Fixes #649