config new --output-dir --push writes a scaffold without the created config ID → duplicates on next sync push

[frantisekrehor]

Summary

config new --output-dir DIR --push creates the remote configuration, then writes a scaffold that does not carry the ID of the config it just created. The next sync push treats that directory as a new configuration and creates a duplicate.

The flag combination is documented as valid ("Scaffold AND remote create" in scaffold-workflow.md and in --help), so this is not user error.

Impact

Real incident on our side: 34 duplicate configurations in a single dev branch. Recovery meant deleting the branch and starting over. The duplicates are not obvious immediately — they appear only after the follow-up sync push, by which point the branch looks fine.

Since then we have banned the combination in our internal docs and use the two-step path (scaffold without --push → edit → sync push) instead.

Root cause (v0.88.0)

commands/config.py:1408 writes the scaffold to disk after the POST succeeds:

if output_dir and scaffold is not None and not dry_run:
    _write_scaffold_to_disk(formatter, scaffold, output_dir, json_mode=False, silent=True)

But scaffold was generated before the POST (config.py:1350), and generate_scaffold() never receives or returns a config ID — services/component_service.py:560-613 takes only alias, component_id, name and derives the directory from slugify(config_name).

The emitted _config.yml therefore contains only:

_keboola:
  component_id: keboola.snowflake-transformation

with the header comment (component_service.py:86):

# NOTE: config_id will be assigned by Keboola on first push

That comment is correct for the scaffold-only path. On the --push path the config already exists, so the "first push" creates a second one.

push_result holds the new ID at that point; it is simply never propagated into the scaffold.

Reproduced on v0.88.0

Fresh dev branch, clean sync diff (unchanged: 39, everything else 0):

$ kbagent --json config new --project P \
    --component-id keboola.snowflake-transformation \
    --name "ZZ Scaffold Push Bug Test" --output-dir . --push
{"status": "ok", "data": {"id": "01m0njbrqwpyqbx0yqfqq9pyen", ...}}

The config now exists remotely. The scaffold written next to it:

# NOTE: config_id will be assigned by Keboola on first push
version: 2
name: "ZZ Scaffold Push Bug Test"
...
_keboola:
  component_id: keboola.snowflake-transformation

No config_id — even though 01m0njbrqwpyqbx0yqfqq9pyen was just created.

sync diff immediately after:

"summary": {"added": 1, "remote_only": 1, "unchanged": 39, ...}

Both halves of the unpaired result are visible at once: the local directory counts as added, and the config that was just created counts as remote_only.

sync push --dry-run confirms what a real push would do:

{"change_type": "added", "config_id": "",
 "config_name": "ZZ Scaffold Push Bug Test",
 "path": "transformation/keboola.snowflake-transformation/zz-scaffold-push-bug-test"}

An added with an empty config_id, for a configuration that already exists — the push creates a second one. (Stopped at --dry-run; the branch and config were deleted afterwards.)

Suggested fix

Pass the created config ID into the scaffold before writing it — either by having _write_scaffold_to_disk accept an override from push_result, or by re-rendering _config.yml with _keboola.config_id filled in on this path. The scaffold-only path should keep today's behaviour and the "assigned on first push" comment.

Environment

kbagent v0.88.0 (code inspected at tag v0.88.0; incident originally hit on an earlier version and nothing in the 0.85–0.88 changelogs addresses it).