fix(sync): reject non-scalar values in sync clone override files - #646
Merged
Conversation
padak
added a commit
that referenced
this pull request
Aug 22, 2026
…rsion Devin review on #646 flagged the '(since vX.Y.Z)' placeholder in the new sync clone override gotcha -- the repo rule requires a concrete version tag, and an identical placeholder from the 0.88.0 cycle (gotchas.md, describe-batch section) shows the 'tag at release' step rots. Bump the version in this PR instead: pyproject 0.88.1 + changelog entry (#646) + version-sync (plugin.json, marketplace.json, uv.lock), and tag the gotcha, sync-workflow note and CLAUDE.md line with 0.88.1.
sync clone's --bucket-map / --variable-values / --instance-rename loader
coerced every YAML value with bare str(value), so a nested mapping (one
fat-fingered colon away from valid input, e.g. 'in.c-old:' followed by an
indented 'new: in.c-new') was silently used as the literal string
"{'new': 'in.c-new'}" -- a bogus bucket ID pushed into the target project.
Extract the load-and-validate logic into a shared yaml_input module
(load_flat_scalar_mapping + yaml_type_name) that rejects non-scalar values
and None with a ConfigError naming the offending key and its actual type in
YAML vocabulary (mapping/list/null), mirroring the describe-batch shape
validation approach from PR #645. Scalars keep the existing str coercion.
commands/flow.py's _load_flow_yaml is deliberately left alone: flow
definitions are legitimately nested, so only the top-level-mapping check is
shared there and it already rejects correctly.
…rsion Devin review on #646 flagged the '(since vX.Y.Z)' placeholder in the new sync clone override gotcha -- the repo rule requires a concrete version tag, and an identical placeholder from the 0.88.0 cycle (gotchas.md, describe-batch section) shows the 'tag at release' step rots. Bump the version in this PR instead: pyproject 0.89.1 (renumbered from 0.88.1 after 0.89.0 landed on main) + changelog entry (#646) + version-sync (plugin.json, marketplace.json, uv.lock), and tag the gotcha, sync-workflow note and CLAUDE.md line with 0.89.1.
…dation Now that #645 is on main, drop _describe_batch_input.py's private _TYPE_NAMES/_type_name copy in favor of the shared yaml_input.yaml_type_name introduced for the sync clone override validation -- the reuse follow-up both PRs promised. Behavior unchanged; the describe-batch error messages keep their per-section subjects and examples.
0.89.0 (config delete trash guard, #643) is merged on main but not yet published, and changelog-check allows exactly one in-flight version -- so the #646 entry joins the 0.89.0 release being prepared instead of stacking a second unreleased bump on top. pyproject back to 0.89.0 (version-sync'd), docs tags now read (since v0.89.0).
padak
force-pushed
the
fix/sync-clone-override-shape-validation
branch
from
August 22, 2026 21:29
1c854db to
9c61b4d
Compare
padak
added a commit
that referenced
this pull request
Aug 23, 2026
…es (#651) Release prep for 0.89.0: adds the changelog entries for everything merged since v0.88.0 (#620, #642, #643, #644, #645, #646, #647, #648, #649, #650, #517, #586, #641), resolves every vNEXT placeholder left by feature PRs to v0.89.0 per the new #648 release process, closes the 10 gaps a full doc-surface audit found across the kbagent plugin (SKILL.md triggers, commands-reference, gotchas, workflow files, keboola-expert.md, AGENT_CONTEXT, CLAUDE.md), and records the live e2e verification evidence. Version files were already at 0.89.0 (bumped by #643); make version-sync is a no-op.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
sync clone's override-file loader (_load_override_fileincommands/sync.py, backing--bucket-map/--variable-values/--instance-rename) coerced every YAML value with barestr(value). A nested mapping — one fat-fingered colon away from valid input:was silently accepted as the literal string
"{'new': 'in.c-new'}"and pushed into the target project as a "bucket ID". No error, no warning — just silently wrong rewrites.Fix
Extract the load-and-validate logic into a small shared module
src/keboola_agent_cli/yaml_input.py:load_flat_scalar_mapping(path, *, label)— loads a JSON/YAML file as a flat{str: str}mapping. Scalar values (string/number/boolean) keep the existingstrcoercion; a mapping, list, ornullvalue is rejected with aConfigErrornaming the offending key and its actual type. Thesync clonecommand already mapsConfigErrortoCONFIG_ERROR+ exit 5, so no new wiring.yaml_type_name(value)— type names in YAML vocabulary (mapping/list/null/...), not Python's — the author is reading their own YAML file, not a traceback.This mirrors the shape-validation approach #645 introduced for
storage describe-batch --from-file.Reuse notes (the three near-identical implementations)
commands/sync.py_load_override_file— now delegates to the shared helper.services/_describe_batch_input.py(fix(storage): reject malformed describe-batch --from-file shapes with a structured error (#640) #645, merged while this PR was open) — now imports the sharedyaml_type_nameand drops its private_TYPE_NAMES/_type_namecopy; its per-section subjects/examples and the nestedcolumnssection stay its own.commands/flow.py_load_flow_yaml— deliberately left alone: flow definitions are legitimately nested, so the flat-scalar contract does not apply; its existing top-level-mapping check already rejects correctly and has no silent-stringification bug.Versioning
0.89.0 (config delete trash guard, #643) is merged on main but not yet published, so this PR's changelog entry is folded into that in-flight 0.89.0 release (changelog-check allows exactly one unreleased version) and the docs tags read
(since v0.89.0)— a concrete tag, per the Devin review finding, instead of a(since vX.Y.Z)placeholder.Tests
tests/test_yaml_input.py(18 tests): valid JSON/YAML, scalar coercion, rejection of nested mapping / list / null values (message names the key and the YAML type), top-level non-mapping, empty file, missing file, invalid YAML, label wording.tests/test_sync_clone.py: a nested--bucket-mapvalue exits 5, names the offending key, and never reachesSyncService.clone_project.Docs:
gotchas.md(sync clone section) +sync-workflow.mdoverride-file notes +CLAUDE.mdsync clone line, all tagged(since v0.89.0).make checkgreen after the rebase onto 0.89.0 main: 5953 passed, 12 skipped.