Skip to content

fix(sync): ignore keboola.mcp-server-tool and honor manifest ignoredComponents - #691

Merged
padak merged 7 commits into
mainfrom
claude/issue-689-pr-7ad2c7
Aug 25, 2026
Merged

fix(sync): ignore keboola.mcp-server-tool and honor manifest ignoredComponents#691
padak merged 7 commits into
mainfrom
claude/issue-689-pr-7ad2c7

Conversation

@padak

@padak padak commented Aug 25, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #689 — both gaps:

  1. keboola.mcp-server-tool is now on ALWAYS_IGNORED_COMPONENTS (constants.py), next to keboola.sandboxes — same category: empty workspace-record configs auto-created by the Keboola MCP server, one per project, managed through separate APIs. sync pull no longer materializes them; diff/push no longer see them.
  2. The manifest field ignoredComponents is now live (it was declared in the schema but read by nothing). A new SyncService._effective_ignored_components() helper unions it with ALWAYS_IGNORED_COMPONENTS, computed once per operation and threaded into every filtering site: the pull fetch loop, the force-pull conflict guard, the diff remote build, scope_manifest (branch scoping, sync: production diff/push after 'sync pull --branch' flags the whole orphaned main/ tree as added (mass-duplicate risk) #649), and the untracked-config/untracked-row walks.

Stale-tree cleanup (the issue's "already-polluted tree" ask)

On the next sync pull, manifest entries for a newly-ignored component are dropped and their local directories removed, reported with the new pull action "ignored" — deliberately distinct from "removed" (which means the config was genuinely deleted on the remote). Human output renders it as its own "Dropped (N) -- component now ignored" section.

One correction to the issue's framing (verified by test)

With the remote side filtered but the local side not, a stale manifest entry does not classify as DELETEDcompute_changeset derives deletes from remote keys. It classifies as added while keeping its existing config id, so every sync push would CREATE a duplicate of a live config. (The DELETED the issue reproduced is the pre-fix behavior where the component was not ignored at all — deleting the dir of a tracked config is a legitimate GitOps delete.) The fix is the same either way: diff filters the local side too, so an ignored component contributes nothing — not added, not deleted, not orphaned.

Changes

  • constants.pykeboola.mcp-server-tool added to ALWAYS_IGNORED_COMPONENTS
  • services/sync_service.py_effective_ignored_components() + threading into pull/diff/conflict-guard/untracked walks; pull's stale-entry sweep distinguishes ignored vs removed
  • sync/branch_scope.pyscope_manifest(..., ignored_components=...) drops ignored entries from every partition
  • commands/sync.py — renders the ignored action (human + one-liner)
  • Docs: gotchas.md, sync-workflow.md (new "Ignored components" section), commands-reference.md, CLAUDE.md, keboola-expert.md, TUTORIAL.md — all (since vNEXT)
  • Tests: TestIssue689IgnoredComponents (8 service tests, TDD-verified failing pre-fix) + 2 CLI rendering tests

No version bump, no changelog entry (feature PR per the release process).

Verification

  • Full suite: 6161 passed, 61 skipped
  • make lint / format-check / skill-check / version-check / version-gate-check / loc-check / check-sentinel-guards all clean
  • Live e2e round-trip against a real project (~/kbagent/e2e/sync_ignored_components_e2e.py): pull-skip, manifest ignore, "ignored" action cleanup, stale-manifest trap (no delete, no duplicate), un-ignore re-materialization

Open in Devin Review

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Open in Devin Review

Comment thread plugins/kbagent/skills/kbagent/references/gotchas.md Outdated
Comment thread plugins/kbagent/skills/kbagent/references/sync-workflow.md Outdated

@padak padak left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of #691 — fix(sync): ignore keboola.mcp-server-tool and honor manifest ignoredComponents

Generated by kbagent-pr-reviewer subagent. Verdict and findings below
are advisory; the human author retains every veto. CI-coverable issues
(lint, format, tests) are confirmed via make check, not duplicated here.

Summary

This PR fixes issue #689 by (1) adding keboola.mcp-server-tool to ALWAYS_IGNORED_COMPONENTS in constants.py, and (2) making the manifest's previously-dead ignoredComponents field live, unioning it with the hardcoded set and threading it through sync pull, sync diff, sync push (which builds on diff), and scope_manifest. It also closes a real "delete-dir-then-push destroys production" trap by filtering the local side of the diff too, and adds a new "ignored" pull action distinct from "removed" for stale-entry cleanup. The change is well-scoped to the sync engine, has strong test coverage (8 service tests + 2 CLI tests, all passing), and make check is fully green on the current PR head (6164 passed, 12 skipped) after a follow-up commit fixed an initial ty typing issue. Verdict: COMMENT — one hand-maintained doc surface (commands/context.py AGENT_CONTEXT) was missed while five sibling doc surfaces were correctly updated.

Verdict

  • Verdict: COMMENT
  • Blocking findings: 0
  • Non-blocking findings: 2
  • Nits: 1

Blocking findings

(none)

Non-blocking findings

[NB-1] src/keboola_agent_cli/commands/context.pyAGENT_CONTEXT sync-pull section not updated for the new "ignored" action / ignoredComponents

kbagent context (the primary reference AI agents load at session start) documents sync pull's output nuances in detail — --force conflict semantics, --theirs, isDisabled round-tripping, renamed-config detection — but says nothing about the always-ignored component set or the new "ignored" pull-action value this PR introduces (services/sync_service.py:1078, commands/sync.py:171). Every other hand-maintained doc surface in the Plugin synchronization map (CLAUDE.md, keboola-expert.md, gotchas.md, sync-workflow.md, commands-reference.md) was correctly updated with a (since vNEXT) tag — this is the one that was missed. An agent parsing --json pull output and seeing action: "ignored" for the first time, or wondering why keboola.mcp-server-tool never shows up in a pulled tree, has no primary-reference answer. Add a short note to the kbagent sync pull entry in commands/context.py mirroring the one already added to commands-reference.md.

[NB-2] PR-description "live e2e round-trip" is not captured as a permanent regression test

The PR description cites a one-off script (~/kbagent/e2e/sync_ignored_components_e2e.py) run against a real project to verify pull-skip / ignore-cleanup / stale-manifest / un-ignore behavior. That script is outside the repo and won't run again on the next release. Since this is a behavior change to existing commands (not a new command), tests/test_e2e.py coverage is not strictly mandated by CONTRIBUTING.md's "every new command" rule, but given the sharp production-delete failure mode this PR closes, a permanent test_e2e.py case (even a light one exercising ignoredComponents end-to-end) would guard against a future regression that the 10 mocked unit tests could miss if a client/API contract shifts. Not a merge blocker given the thorough mock-level coverage.

Nits

  • [NIT-1] src/keboola_agent_cli/services/sync_service.py:1074-1080 — the "ignored" if old_cfg.component_id in ignored_components else "removed" conditional is doing meaningful classification work inline inside a dict literal; a tiny named helper (e.g. _classify_stale_entry(component_id, ignored_components) -> str) would read slightly cleaner and give the branch a name, though the accompanying comment already makes the intent clear.

Verification log

  • gh pr view 691 --json title,body,files,additions,deletions,baseRefName,headRefName,labels,state → OPEN, fix(sync): prefix matches the change type (bug fix to existing sync commands), 12 files changed, +579/-14 (initial), +585/-14 after the follow-up ty fix commit ✓
  • git rev-parse --abbrev-ref HEADclaude/issue-689-pr-7ad2c7, matches <branch> input; working tree confirmed on the PR branch throughout ✓
  • gh pr diff 691 → 865 lines, re-fetched after a follow-up commit landed on the PR mid-review (see below) ✓
  • 3-layer compliance: grep for typer/click/formatter/console.print in services/, httpx/requests in commands/ → all empty, no layer violations ✓
  • Plugin synchronization map walk: no new CLI command in this PR (behavior fix to existing sync pull/diff/push), so permissions.py OPERATION_REGISTRY, server/routers/sync.py (sync has no REST router — filesystem-local per CLAUDE.md), and the SKILL.md decision table are unaffected. Confirmed plugins/kbagent/agents/keboola-expert.md §3 gotcha added (no new command group, so §2 matrix correctly untouched), gotchas.md new entry tagged (since vNEXT), commands-reference.md updated for sync pull/push/diff, sync-workflow.md new "Ignored components" section, CLAUDE.md ## All CLI Commands inline comment updated. commands/context.py AGENT_CONTEXT NOT updated — see NB-1.
  • Convention checks (magic numbers, raw error_code= strings, bare except:, print() in prod code, token leaks, new tuple[...] returns): all grep patterns from the playbook → empty on the added lines ✓
  • make check — first run (against the PR's first commit 152ed5a5 only) failed at typecheck: 4 ty diagnostics, unresolved-attribute on SyncService._test_client in tests/test_sync_service.py (mypy-style # type: ignore[attr-defined] comments do not suppress ty errors; CONTRIBUTING.md requires # ty: ignore[rule]). A second commit (7dde1a3a, "test(sync): satisfy ty -- return the mock client from the helper instead of stashing it on SyncService") landed on the PR branch mid-review and fixes exactly this, refactoring _svc into _svc_with_client returning tuple[SyncService, MagicMock] instead of monkeypatching an untyped attribute. Re-ran make check against the current PR head: exit 0, all gates greenruff check, ruff format --check, ty check (0 diagnostics besides the pre-existing unrelated hatchling warning), check_version_gates.py, check_command_sync.py, generate_changelog.py --check, check_error_codes.py, check_sentinel_guards.py, check_file_size.py, and pytest tests/ -m "not e2e"6164 passed, 12 skipped. No BLOCKING finding remains for make check.
  • Test coverage: tests/test_sync_service.py::TestIssue689IgnoredComponents → 8 service-layer tests (pull-skip, manifest-honor, ignored-vs-removed classification, force-pull conflict guard, diff exclusion both sides, push no-op, untracked-directory guard); tests/test_sync_cli.py → 2 CLI-layer tests (human-mode "Dropped ... component now ignored" rendering, _pull_one_liner ignored count). Matches the PR description's claimed counts exactly ✓
  • Backward compatibility: new pull-action value "ignored" is additive to the details[].action JSON field; grep'd web/frontend/src for any consumer keying off pull action values → none found (Agents.tsx/Lineage.tsx hits were unrelated "pull" matches). No breaking rename/removal of existing fields. scope_manifest()'s new ignored_components parameter has a safe default (frozenset()) and its single call site (services/sync_service.py:1304) was updated ✓
  • Could not reproduce the live round-trip against a real Keboola project myself (no credentials in this review environment; per project convention, AI agents never handle API tokens) — the PR author states a live e2e round-trip was already run manually; the unit-level mock tests independently exercise the same scenarios and all pass.

Open questions for the author

(none)

padak added a commit that referenced this pull request Aug 25, 2026
- NB-1: document ignored components + the new 'ignored' pull action in
  commands/context.py AGENT_CONTEXT (sync pull + sync diff entries), the
  one silent-drift surface the PR missed.
- NB-2: add test_sync_ignored_components_round_trip to tests/test_e2e.py
  (TestE2ESyncWorkflow) so the ignoredComponents round-trip -- including
  the stale-manifest trap -- is exercised permanently by the nightly E2E
  run, not just by a one-off out-of-repo script.
- NIT-1: name the stale-entry classification branch (stale_action) in
  SyncService.pull.
@padak

padak commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

Addressed all review findings:

  • NB-1 (fd918600): commands/context.py AGENT_CONTEXT now documents ignored components -- the always-ignored set + ignoredComponents union on the sync pull entry (including the new details[].action "ignored" vs "removed" distinction) and the both-sides filtering note on the sync diff entry.
  • NB-2 (fd918600): added TestE2ESyncWorkflow::test_sync_ignored_components_round_trip to tests/test_e2e.py -- a permanent nightly-E2E case covering materialize -> ignore+pull cleanup (action: "ignored") -> un-ignore re-materialize -> the stale-manifest trap (re-ignore without pull + hand-deleted dir: no deleted diff entry, no orphaned, push --dry-run plans nothing, remote config verified untouched).
  • NIT-1 (fd918600): the stale-entry classification is now a named stale_action variable.
  • Devin findings (0af50717): moved (since vNEXT) out of the two new markdown headings (plus the pre-existing serve: honor the root-level --config-dir, and fix two stale permissions comments #681 one) per the CONTRIBUTING anchor-slug rule; threads replied + resolved.

make check-equivalent gates re-verified locally after the changes (ruff, ty, command-sync, version-gate, sync test suite).

@padak
padak force-pushed the claude/issue-689-pr-7ad2c7 branch from fd91860 to d42e5c8 Compare August 25, 2026 13:03
padak added a commit that referenced this pull request Aug 25, 2026
- NB-1: document ignored components + the new 'ignored' pull action in
  commands/context.py AGENT_CONTEXT (sync pull + sync diff entries), the
  one silent-drift surface the PR missed.
- NB-2: add test_sync_ignored_components_round_trip to tests/test_e2e.py
  (TestE2ESyncWorkflow) so the ignoredComponents round-trip -- including
  the stale-manifest trap -- is exercised permanently by the nightly E2E
  run, not just by a one-off out-of-repo script.
- NIT-1: name the stale-entry classification branch (stale_action) in
  SyncService.pull.
padak added 7 commits August 25, 2026 15:09
…omponents (#689)

Two related gaps in the GitOps sync engine:

- keboola.mcp-server-tool workspace records (empty configs auto-created by
  the Keboola MCP server, one per project) joined ALWAYS_IGNORED_COMPONENTS
  next to keboola.sandboxes: same category, managed through separate APIs.
- The manifest field ignoredComponents was declared in the schema but read
  by nothing. It is now live: unioned with ALWAYS_IGNORED_COMPONENTS and
  honored by sync pull, diff and push (push builds on diff), plus the
  force-pull conflict guard.

Stale-tree cleanup: on the next pull, manifest entries for a newly-ignored
component are dropped and their local directories removed, reported with the
new pull action "ignored" (distinct from "removed" = deleted on remote).
The diff also filters its LOCAL side by the ignored set (manifest entries,
untracked-config walk, untracked-row walk), so a stale entry or leftover
directory for an ignored component contributes nothing to the changeset --
without that, a stale entry with no remote counterpart classified as
"added" and every push CREATEd a duplicate of a live config.

Closes #689
…ead of stashing it on SyncService

ty does not honor mypy-style type: ignore comments, so the dynamic
_test_client attribute failed CI's type-check step. Also corrects the
test-class docstring to the verified failure mode: a stale local entry
with a filtered remote counterpart classifies as 'added' (duplicate
create on every push), not 'deleted'.
Moves the (since vNEXT) tags from three section headings -- the two new
ignored-components sections plus the pre-existing serve --config-dir
gotcha from #681 -- to the sections' first body line, so the heading
anchor slugs stay stable when the placeholder resolves at release.
- NB-1: document ignored components + the new 'ignored' pull action in
  commands/context.py AGENT_CONTEXT (sync pull + sync diff entries), the
  one silent-drift surface the PR missed.
- NB-2: add test_sync_ignored_components_round_trip to tests/test_e2e.py
  (TestE2ESyncWorkflow) so the ignoredComponents round-trip -- including
  the stale-manifest trap -- is exercised permanently by the nightly E2E
  run, not just by a one-off out-of-repo script.
- NIT-1: name the stale-entry classification branch (stale_action) in
  SyncService.pull.
@padak
padak force-pushed the claude/issue-689-pr-7ad2c7 branch from 3f2e62b to 9e35174 Compare August 25, 2026 13:10
@padak
padak merged commit f6eeb45 into main Aug 25, 2026
3 checks passed
@padak
padak deleted the claude/issue-689-pr-7ad2c7 branch August 25, 2026 13:10
padak added a commit that referenced this pull request Aug 25, 2026
Resolves the #691 (ignoredComponents) overlap: _is_conflict and
_detect_force_pull_conflicts moved to _sync_baseline.py by this branch,
so main's ignored_components parameter and filter are carried into the
moved detect_force_pull_conflicts; _effective_ignored_components stays
on SyncService. Doc conflicts combine both sides.
padak added a commit that referenced this pull request Aug 25, 2026
Moves the (since vNEXT, #687) tag from the workspace-workflow.md load-types heading onto the section's first body line, keeping the heading anchor slug stable when the placeholder resolves at release (CONTRIBUTING rule). The gotchas.md portion of this PR was already applied identically by the merged #691, so it lands as a no-op.
@padak padak mentioned this pull request Aug 25, 2026
13 tasks
padak added a commit that referenced this pull request Aug 25, 2026
Bump pyproject to 0.91.0, add the changelog entry covering every PR merged since v0.90.1 (#627, #681, #691, #692, #693, #694, #695, #696, #697, #698), resolve all 54 vNEXT version-gate placeholders, and run version-sync + skill-gen.

No web/frontend changes in this batch, so no whatsnew.ts entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sync: ignore keboola.mcp-server-tool workspace records like keboola.sandboxes — and manifest ignoredComponents is never read

1 participant