sync git-branching: branch_id type confusion + walker scope guard block GitOps workflow

[padak]

TL;DR

Two chained bugs in kbagent sync make GitOps workflows unusable on gitBranching.enabled: true workspaces:

• Bug A — kbagent sync pull mis-routes dev-branch configs into main/ and corrupts .keboola/manifest.json by appending a duplicate branches[] entry on every invocation. Root cause: Keboola branch IDs are persisted as str in branch-mapping.json but compared as int everywhere else, so every cross-type == lookup silently fails.

• Bug B — kbagent sync diff and kbagent sync push --dry-run silently report "no changes" when the user scaffolds a brand-new local config dir under a non-default branch path. The "scaffold locally then push" flow advertised by the docs is unreachable on git-branching workspaces.

The two bugs are independent at the symptom level but Bug A blocks a complete fix for Bug B (the path-resolution helper that Bug B's walker delegates to is itself broken by Bug A). They should be fixed together in one PR.

Reported externally against v0.27.0; reproduced in code review on v0.30.3 (current main). Both bugs are still latent.

---

Shared environment

• Manifest schema v3, gitBranching.enabled: true, defaultBranch: main.
• Active git feature branch linked via kbagent sync branch-link --branch <DEV_BRANCH_ID> to a non-default Keboola dev branch.
• Manifest pre-state:

  "branches": [
    { "id": 38516, "path": "main",         "metadata": {} },
    { "id": 38721, "path": "branch-38721", "metadata": {} }
  ]

---

Bug A — sync pull mis-routes to main/ and inflates branches[]

Symptom

Single warning line on stderr:

WARNING Branch ID 38721 not found in manifest, falling back to default path

— even though {"id": 38721, "path": "branch-38721"} is plainly present in manifest.branches[]. The lookup fails, the fallback writes to main/, and a separate code path then re-registers the "missing" branch under a mangled path.

Repro

kbagent --json sync pull --project <alias>

1. The warning above is emitted.
2. JSON response indicates a successful pull, but branch_dir: "main":

   { "status": "ok",
     "data": {
       "status": "pulled",
       "branch_id": "38721",
       "branch_dir": "main",
       "configs_pulled": 1,
       "details": [
         { "action": "updated",
           "component_id": "<component-id>",
           "path": "application/<component-id>/<config-name>" } ] } }

3. Files appear on disk at main/application/<component-id>/<config-name>/ instead of branch-38721/....
4. manifest.branches[] grows by one entry per pull:

   { "id": 38721, "path": "branch-38721-38721", "metadata": {} }

   Each subsequent sync pull appends another duplicate (branch-38721-38721-38721, etc.).
5. manifest.configurations[] records "branchId": 38721 with the correct relative path for the dev branch — so the manifest now misdescribes where files actually live.

Root cause — branch_id is str in branch-mapping.json, int everywhere else

branch_link converts the Keboola branch ID to a string before persisting it. services/sync_service.py lines 1957, 1967, 1971, 1981, 1985:

kbc_branch_id = str(branch_info["id"])   # all five branches of branch_link

That string lands on BranchMappingEntry.keboola_id, whose annotation in sync/branch_mapping.py:19 is already str | None — the bug is locked in by the type annotation.

_resolve_branch_id (services/sync_service.py:2085) is annotated -> int | None but actually returns the string straight from the mapping (line 2111: return entry.keboola_id). The annotation lies.

pull() (line 280) then calls _ensure_branch_registered(manifest, "<id-as-string>", client) and _find_branch_path(manifest, "<id-as-string>"). Both compare:

if branch.id == branch_id:

…where branch.id is int (from the Pydantic ManifestBranch.id: int at sync/manifest.py:67) and branch_id is str. Python returns False for cross-type == between int and str, so:

• _ensure_branch_registered (line 2548) thinks the branch is unregistered → falls through to the registration block → path-collision logic (line 2586) sees branch-38721 is taken → suffixes to branch-38721-38721 → manifest.branches.append(...). Pydantic coerces the string id back to int on append, so the entry lands as {id: 38721, path: "branch-38721-38721"}. The same (id, path) mutation runs on every pull.
• _find_branch_path (line 2593) hits the same False comparison → falls back to manifest.branches[0].path → returns "main". Files get routed to main/.
• The branch-name lookup inside _ensure_branch_registered (line 2575: if b.get("id") == branch_id) also fails for the same reason — Storage API returns id as int, our branch_id is str — so the human-readable name from the API never gets used and we always fall through to f"branch-{branch_id}". This explains why naming.branch = "{branch_name}" produced branch-38721 instead of a slug.

So one type bug produces three visible symptoms: misrouted files, manifest growth, and missing branch slug.

Existing test that locks the bug in

tests/test_sync_service.py:1613:

assert entry.keboola_id == "99999"

This assertion encodes the wrong contract. After the fix it must change to == 99999 (int).

---

Bug B — sync diff / sync push --dry-run does not detect untracked local configs in non-default branch

Symptom

kbagent --json sync diff --project <alias>

{ "status": "ok",
  "data": {
    "changes": [], "remote_only": [],
    "summary": { "added": 0, "modified": 0, "remote_modified": 0,
                 "conflict": 0, "deleted": 0, "unchanged": 0,
                 "remote_only": 0 } } }

kbagent --json sync push --project <alias> --dry-run

{ "status": "ok",
  "data": { "status": "no_changes",
            "created": 0, "updated": 0, "deleted": 0, "errors": [] } }

Per skills/kbagent/references/sync-workflow.md — | ADDED | New local config | Push creates it | — an untracked local config dir is supposed to be surfaced as diff state "added" and POSTed by push. The "scaffold locally then push" workflow is silently broken.

Repro

1. kbagent sync init --git-branching against an empty (or near-empty) project — manifest's configurations: [] is empty.
2. kbagent sync pull --project <alias> (no-op against an empty project, but auto-registers default branch).
3. On a feature git branch, kbagent sync branch-link --branch <DEV_BRANCH_ID>.
4. Manually create a config directory matching default naming under the dev-branch path:

   branch-<id>/application/<component-id>/<config-name>/
     _config.yml
     code.py
     pyproject.toml
   

5. Run kbagent sync diff and kbagent sync push --dry-run → both return empty. No way to push the new config to Keboola via sync.

Root cause — walker piggybacks on manifest.configurations

services/sync_service.py:_find_untracked_configs (line 2612) builds the set of branch IDs to scan exclusively from already-tracked configurations:

for cfg in manifest.configurations:
    branch_path = self._find_branch_path(manifest, cfg.branch_id)
    tracked_paths.add(str(project_root / branch_path / cfg.path))
    active_branch_ids.add(cfg.branch_id)
...
for branch in manifest.branches:
    # Only scan branches that have tracked configs — skip inactive
    # branch directories to avoid phantom "added" configs.
    if branch.id not in active_branch_ids:
        continue

When manifest.configurations: [], active_branch_ids is empty → the continue skips every branch → no directory is ever walked → added: [].

The guard ("skip inactive branch directories to avoid phantom adds") is legitimate for orphaned dev-branch dirs left over from old work, but its scope is too narrow: it also blocks the documented first-time scaffold flow. There is no way to bootstrap a config locally through sync push against a clean configurations: [] manifest, contradicting the sync-workflow.md docs.

Why this fix depends on Bug A

Even after _find_untracked_configs is widened, services/sync_service.py:906 (in diff()) computes the path for newly-found untracked configs as:

branch_path = self._find_branch_path(manifest, branch_id)

…where branch_id is the value returned by _resolve_branch_id, which today returns a str due to Bug A. _find_branch_path then fails the branch.id (int) == branch_id (str) comparison and falls back to "main", so even with a wider walker scope, untracked configs found in branch-38721/ would be reported with path rooted at main/. Bug A must be fixed first or simultaneously.

---

Proposed fix (single PR)

Bug A — make branch IDs int end-to-end

1. services/sync_service.py lines 1957, 1967, 1971, 1981, 1985: drop the str(...) wrapper:

   kbc_branch_id = int(branch_info["id"])

2. sync/branch_mapping.py line 19: change annotation keboola_id: str | None → keboola_id: int | None. Update to_dict() / from_dict() accordingly. from_dict should coerce entry.get("id") to int (with None handling) so any legacy branch-mapping.json files written by older kbagent versions auto-migrate on first read — no breaking change for users who already linked branches.
3. tests/test_sync_service.py line 1613: update the asserted type to int (== 99999).
4. (Defense in depth) Make _ensure_branch_registered idempotent on (id, path) even after the type fix so any future regression here doesn't silently mutate the manifest.

Bug B — widen walker scope without re-opening the phantom-add hole

Replace the active_branch_ids guard in _find_untracked_configs with a wider source set:

• All branch IDs in manifest.configurations[] (today's set — keeps protection against orphaned dirs that were once tracked then deleted).
• The branch ID currently resolved by _resolve_branch_id() for this pull / diff / push call — the user is on a linked git branch and explicitly asked to operate against it.
• The defaultBranch ID from manifest.git_branching (always allowed — push to main is a legitimate first-time flow too).

Pass the resolved branch_id into _find_untracked_configs(...) from diff() and push() so the walker has the explicit signal. Phantom-add protection is preserved for dirs from random other branches no one is currently working on.

Regression tests

• Bug A: pre-populated manifest with branches = [{id: 38516, path: "main"}, {id: 38721, path: "branch-38721"}], mocked branch-mapping.json with keboola_id set, run pull twice; assert len(manifest.branches) == 2 after both pulls and branch_dir == "branch-38721" in the response.
• Bug B: manifest.configurations: [], freshly scaffolded dir under a non-default branches[].path, expect diff to report added: 1 and push --dry-run to report created: 1.

---

Asks for the maintainer

1. Confirm the intended contract for BranchMappingEntry.keboola_id is int (and only the persisted JSON is string-of-int) or that it should be string-typed throughout. The fix above assumes the former.
2. Decide on legacy-branch-mapping.json migration strategy — silent coerce on load (the simplest), or explicit "manifest schema upgrade" message.
3. Confirm whether ADDED → push creates it is an intended supported flow on git-branching workspaces today. The docs (sync-workflow.md) imply yes; current behavior says no. If the answer becomes "no, must use tool call create_config first", the docs need a clear caveat and a hint on sync push --dry-run like:

   { "warning": "Found 1 local directory matching naming convention but not in manifest; create via 'kbagent tool call create_config' first." }

4. The current Bug A fall-through emits a WARNING and continues with a wrong-branch write — consider promoting to an error with non-zero exit even after the type fix, since "I don't recognize this branch" is not a recoverable state for sync pull.

Workaround (until fix)

For Bug A: after every sync pull, manually move main/<config-path>/* to branch-<id>/<config-path>/ and hand-edit .keboola/manifest.json to remove the {"id": <id>, "path": "branch-<id>-<id>..."} entries from branches[].

For Bug B: bootstrap the config via the MCP tool, then let sync track it:

kbagent --json tool call create_config --project <alias> --branch <DEV_BRANCH_ID> --input @stub.json
kbagent sync pull --project <alias> --force
# Now sync push detects MODIFIED on edits as expected.

This works but defeats the GitOps "scaffold locally, then push" flow that the docs advertise.

[padak]

Update — E2E reproduction + scope expansion (5 chained bugs total)

E2E reproduced both bugs end-to-end against a real Keboola project on our internal kbagent-e2e test stack. A pre-emptive audit of the surrounding code then surfaced three additional latent bugs that share the same root cause or live in the same neighborhood. Updating scope: one PR for all five.

E2E confirmation (run on kbagent v0.30.3)

branch-mapping.json written by sync branch-link (smoking gun for Bug A):

{
  "version": 1,
  "mappings": {
    "main":               { "id": null,      "name": "Main" },
    "feature/issue-267":  { "id": "1296400", "name": "..." }
  }
}

The branch ID is persisted as a string. Every downstream branch.id == branch_id (int == str) comparison is therefore False.

sync pull JSON response (after sync branch-link to dev branch 1296400):

{
  "status": "pulled",
  "branch_id": "1296400",       // STRING in API response
  "branch_dir": "main",         // routed to default branch path
  "configs_pulled": 1
}

with stderr: WARNING Branch ID 1296400 not found in manifest, falling back to default path.

Manifest branches[] after 1 / 2 / 3 successive pulls:

After pull 1: 2 entries — main, branch-1296400                      (correct so far)
After pull 2: 3 entries — main, branch-1296400, branch-1296400-1296400
After pull 3: 4 entries — main, branch-1296400, branch-1296400-1296400, branch-1296400-1296400  ← duplicate (id, path)

Subtle correction to the original report: the third pull does NOT produce branch-1296400-1296400-1296400. The collision-suffix loop in _ensure_branch_registered (line 2586) appends -{branch_id} only once, regardless of how many existing collisions there are. Each pull from #2 onwards appends an identical (id, path) duplicate. This makes the manifest state corruption worse than originally described — multiple entries with the same (id, path) pair, not a chain of progressively longer suffixes.

Bug B confirmation: clean sandbox, manifest.configurations: [], scaffold at branch-1296400/application/test-component/my-test-config/_config.yml:

kbagent --json sync diff --project kbagent-e2e
# → {"changes": [], "summary": {"added": 0, ...}}
kbagent --json sync push --project kbagent-e2e --dry-run
# → {"status": "no_changes", "created": 0, ...}

The walker silently drops the scaffold. Confirmed e2e.

---

Bug C — branch_switched always True on git-branching pulls (HIGH)

services/sync_service.py:489-491:

branch_switched = existing_branch_ids.get(lookup_key, branch_id or 0) != (branch_id or 0)

existing_branch_ids is dict[str, int] (line 356), built from c.branch_id which is the int field on ManifestConfiguration.branch_id: int (sync/manifest.py:94). After Bug A, the right-hand branch_id is str. The comparison int != str is always True for any value, so branch_switched is always True for previously-tracked configs in git-branching mode. Every pull then re-writes every config file even when nothing changed.

E2E confirmation: every successive pull in our reproduction reported files_written: 1 for an unchanged mcp-server-tool config. The "incremental" guard is fully defeated. Pull is the only sync operation where this matters, but it's the most-used one.

This is fixed automatically by the type fix for Bug A — no separate code change needed, just a regression test.

---

Bug D — kbagent branch delete does not clean up branch-mapping.json (HIGH)

services/branch_service.py:274-281: delete_branch deletes the Keboola branch via the Manage API and resets project.active_branch_id to None. It does NOT touch the local .keboola/branch-mapping.json. After deletion, the mapping still claims feature/<git-branch> → 1296400, but Keboola branch 1296400 no longer exists. Every subsequent sync pull / sync push from that git branch hits a 404 from the Storage API. The user is stuck in a non-recoverable state until they manually run kbagent sync branch-unlink.

The same gap applies to kbagent branch merge. After merging a dev branch back, the mapping persists pointing at the (possibly now-deleted) source branch.

sync-workflow.md says "Step 4: kbagent sync branch-unlink", but it is documented as after branch delete, by which point the user is already broken. The fix should make branch delete and branch merge call sync branch-unlink as a side effect (or hint at the workflow inversion).

---

Bug E — Lost branch-mapping.json blocks even main with no recovery (HIGH)

services/sync_service.py:2103-2118: in git-branching mode, _resolve_branch_id requires a mapping entry for the current git branch. The mapping file is auto-created on sync init with a main → None entry, but if the file is later lost (accidental delete, .gitignore drift, manual edit), even checking out main raises ConfigError("Git branch 'main' is not linked"). There is no recovery path: branch_link (line 1918) explicitly forbids linking the default branch, so the user can't even use the documented fix-it command.

The fix should auto-recreate the main → None entry on load if the file is missing or doesn't contain the default branch, and emit an informational message rather than refusing to operate.

---

Medium / lower findings (in same neighborhood)

• Finding 1.1 (M): services/sync_service.py:906 — _find_untracked_configs returns paths relative to a branch dir; the caller in diff() re-resolves through _find_branch_path(manifest, branch_id). With Bug A's str-poisoned branch_id, untracked detection routes through "main" too. Compounds Bug B.
• Finding 4.1 (M): kbagent branch use --branch ID setting is silently ignored when manifest.git_branching.enabled = True. _resolve_branch_id line 2103 short-circuits to the mapping path. Should at minimum warn the user, or merge the two sources of truth.
• Finding 5.3 (M): kbagent branch merge shares Bug D's cleanup gap.
• Finding 7.1, 7.2 (M): sync-workflow.md documents workflows that are broken today (Bug A breaks pull, Bug B breaks scaffold-then-push). Doc updates required after fix.
• CLI inconsistency (L): sync branch-link --branch-id ID vs branch delete --branch ID use different flag names for the same concept. Confused our own reproduction script.

---

Test coverage gaps that would have caught these

tests/test_sync_service.py has 91KB of tests but none of these:

1. No test for branch_link → pull → assert len(manifest.branches) stable across N pulls. Would have caught Bug A's manifest growth. TestBranchLink tests only the link step; TestEnsureBranchRegistered tests _ensure_branch_registered in isolation with branch_id passed as int directly, bypassing the broken _resolve_branch_id.
2. No test for _find_untracked_configs with manifest.configurations: [] and a non-default branch in branches[]. Would have caught Bug B.
3. No test that asserts files land under feature-<id>/ (not main/) when pulling on a linked git branch. Would have caught Bug A's misroute.

Existing tests/test_sync_service.py:1613 (assert entry.keboola_id == "99999") explicitly locks the bug in by encoding the wrong contract.

---

Updated PR scope

Single PR covering:

1. Bug A — type fix end-to-end. int(branch_info["id"]) in branch_link (5 sites), BranchMappingEntry.keboola_id: int | None, from_dict coerces legacy str values to int (silent migration), _resolve_branch_id annotation matches reality.
2. Bug B — walker scope widening. _find_untracked_configs accepts an explicit resolved-branch-id from the caller; scans tracked_configs ∪ {resolved} ∪ {default}; preserves orphan-dir protection.
3. Bug C — auto-fixed by Bug A. No separate code change; add a regression test.
4. Bug D — cleanup hook. branch delete and branch merge call sync_service.branch_unlink if a matching mapping exists; document the workflow change.
5. Bug E — auto-recovery for missing default-branch mapping. _resolve_branch_id (or load_branch_mapping) seeds main → None if missing; informational message, not error.
6. Tests — fill the three gaps above + update test_sync_service.py:1613 to int.
7. Docs — sync-workflow.md: clarify scaffold-locally-then-push works, document branch delete/merge cleanup behavior, fix CLI flag examples.

Skipping (separate issues): CLI flag harmonization (--branch vs --branch-id) and branch use vs git-branching source-of-truth merge — both are UX cleanups, not bugs.

---

E2E reproduction logs (431 lines) are kept locally for the regression-test design phase and will inform the test fixtures. Will reference back to this issue from the eventual PR.

[padak]

PR #268 opened with the full fix for all five chained bugs and after-fix E2E verification against the same internal kbagent-e2e project that confirmed the bugs in the previous comment.

After-fix evidence (kbagent v0.30.4 on the PR branch)

branch-mapping.json written by sync branch-link:

{
  "version": 1,
  "mappings": {
    "main":                          { "id": null,    "name": "Main" },
    "feature/issue-267-after-fix":   { "id": 1296401, "name": "kbagent-issue-267-after-fix-..." }
  }
}

Note id is now a JSON number, not a string — Bug A root cause closed at the persistence layer.

sync pull JSON (3× consecutive on a linked dev branch):

Pull 1: branch_id: 1296401 (int),  branch_dir: "kbagent-issue-267-after-fix-...",  files_written: 3
Pull 2: branch_id: 1296401 (int),  branch_dir: "kbagent-issue-267-after-fix-...",  files_written: 0
Pull 3: branch_id: 1296401 (int),  branch_dir: "kbagent-issue-267-after-fix-...",  files_written: 0

No WARNING Branch ID ... not found in manifest, falling back to default path ever fires.

manifest.branches[] after 3 pulls: 2 entries, stable (was 4 before the fix). No branch-<id>-<id> mangled paths. Branch path is the API-provided sanitized name (Bug A's "side observation" also fixed transitively).

sync diff against scaffolded local config under branch-1296401/:

{
  "changes": [{
    "change_type": "added",
    "config_name": "my-test-config",
    "path": "application/test-component/my-test-config"
  }],
  "summary": { "added": 1, ... }
}

sync push --dry-run: returns status: "dry_run" with the scaffold in changes (was status: "no_changes" pre-fix).

branch delete JSON:

{
  "branch_id": 1296401,
  "was_active": true,
  "message": "Branch ID 1296401 deleted from project 'kbagent-e2e'. Active branch reset to main. Unlinked git branch(es): feature/issue-267-after-fix.",
  "mapping_cleanup": {
    "project_root": "/private/tmp/kbagent-e2e/sync-repro",
    "git_branches_unlinked": ["feature/issue-267-after-fix"]
  }
}

branch-mapping.json after delete: only the main entry remains. No stale 404-bound mapping.

Bug-by-bug status

Bug	Status	Verified by
A — type confusion → misroute + manifest growth	Fixed	E2E (above) + 7 regression tests
B — walker scope guard too narrow	Fixed	E2E (above) + 2 regression tests
C — branch_switched always True	Fixed transitively (Bug A type fix)	E2E (files_written ticks down to 0) + regression test
D — branch delete/merge orphans mapping	Fixed	E2E (above) + 3 regression tests
E — lost mapping blocks default branch	Fixed	2 unit tests (_resolve_branch_id direct)

Out of scope (follow-ups)

Per the audit comment, these are real but separate from issue #267:

• CLI flag harmonization (sync branch-link --branch-id vs branch delete --branch) — UX cleanup.
• kbagent branch use is silently ignored when gitBranching.enabled = True (Finding 4.1) — needs a design decision (warn vs merge sources of truth).
• sync-workflow.md doc updates to reflect that ADDED → push creates it works correctly now — docs-only follow-up.

I'll cherry-pick those into separate issues once this PR lands.