feat(0.31.1): config new --push for one-shot remote create

[ottomansky]

Summary

Extends kbagent config new with a --push flag that POSTs to the Storage API in one CLI call. Closes the F2 gap from kbagent-feature-gaps.md — previously the only "FIIA HTTP call → kbagent CLI" mapping without a first-class command.

• Without --push: byte-for-byte preserved (verified by unchanged TestConfigNew suite).
• With --push: scaffolds + POSTs to /v2/storage/components/{cid}/configs. Returns the new config ID.
• FIIA-style one-shot: --push --no-files skips the filesystem entirely.

kbagent config new --component-id keboola.ex-http --name "API ingest" \
  --project prod --push --no-files                 # one-shot, no files
kbagent config new --component-id keboola.python-transformation-v2 \
  --name "T1" --project prod --push --output-dir ./scratch   # scaffold + POST
kbagent config new --component-id keboola.ex-http --name "smoke" \
  --project prod --push --no-files --dry-run       # preview + validation

Why --push and not a new config create command

The PR-B spec originally proposed a new config create command. After plan-mode discussion we extended config new instead:

1. No mental-model fragmentation. A separate config create would have meant two commands with overlapping intent ("make a new config") but different mechanics. The AI agent docs already conflate the two in places.
2. Reuse of the scaffold pipeline. With --push --output-dir, the same scaffold step that powers scaffold-only mode also produces the files alongside the POST — the GitOps-plus-ID-claim workflow.
3. Zero regression surface. Without --push, today's behavior is byte-for-byte preserved.
4. One-flag flip for FIIA. --push --no-files is the FIIA path: no filesystem state, single HTTP POST, returns the new config ID. Direct replacement for the tool call create_config escape hatch, with the bonus that it does not inherit the MCP keboola.snowflake-transformation refusal (the typed CLI wraps the raw Storage API).

The Snowflake-transformation row in keboola-expert.md is also updated: previously the first-choice was config new + config update --set (two-step without remote create in step 1); now config new --push --no-files works in one call for all component types including Snowflake.

Schema validation

• Validation runs by default when --push is set AND an explicit --configuration / --configuration-file body is given.
• On mismatch: ConfigError → exit 5, error paths listed.
• AI Service has no schema, returns an error, or has a malformed schema → silently skip (validation_status: "skipped"); never blocks creation.
• --no-validate opts out entirely.
• The default empty-shell case (no body passed, POST {}) auto-skips validation — most component schemas require parameters, so the FIIA empty-shell pattern shouldn't be blocked by predictable failures.
• --dry-run returns the validation outcome in the envelope but does NOT abort on validation failure (dry-run is inspection-only).

Behavior matrix

flags	scaffold to stdout	write files	API POST
(none, no --push)	yes (today)	no	no
--output-dir D	no	yes (today)	no
--push --project P --name N	yes	no	yes
--push --project P --name N --output-dir D	no	yes	yes
--push --project P --name N --no-files	no	no	yes

Files changed (18)

Service — services/config_service.py: new create_config() + _validate_config_body() helper. ConfigService.__init__ overridden to accept an ai_client_factory for testability.

Command — commands/config.py: extend config_new() with 8 new options + flag-combination validation block. Refactored the disk write + the human-mode push-result renderer into shared helpers.

Hints — hints/definitions/config.py: new config.new CommandHint (only emitted when --push is set).

Tests

• tests/test_config_create_service.py (new) — 15 cases: happy paths, dry-run, schema validation (ok / failed / skipped / no-schema / AI error / malformed schema / validate=False), client cleanup on error.
• tests/test_config_create_cli.py (new) — 20 cases: flag-combination validation (9), push-mode happy paths (8), error propagation (3).
• tests/test_e2e.py — new step 19b: dry-run → real create → detail verify → update --set patch → delete cleanup, wrapped in try/finally with pre-registered safety-net teardown.

Sync map (CLAUDE.md convention #17 — all 7 surfaces updated)

• src/keboola_agent_cli/commands/context.py — inventory + usage note
• CLAUDE.md ## All CLI Commands — replaced config new inventory line
• plugins/kbagent/agents/keboola-expert.md — amended Snowflake matrix row + amended inline Snowflake gotcha + new "Create a new config (one-shot remote)" row
• plugins/kbagent/skills/kbagent/SKILL.md — auto-regenerated via make skill-gen
• plugins/kbagent/skills/kbagent/references/commands-reference.md — two-mode entry with full flag inventory
• plugins/kbagent/skills/kbagent/references/gotchas.md — new (since v0.31.1) section
• plugins/kbagent/skills/kbagent/references/scaffold-workflow.md — dual-mode callout + new push examples in step 3

Version

• pyproject.toml: 0.31.0 → 0.31.1 (patch — single command extension, no breaking changes)
• src/keboola_agent_cli/changelog.py: new 0.31.1 entry
• plugins/kbagent/.claude-plugin/plugin.json, .claude-plugin/marketplace.json: synced via make version-sync
• uv.lock: refreshed via uv sync

Test plan

• make check clean — ruff lint, ruff format-check, skill-check, version-check, changelog-check, 2919 tests pass
• Service tests: 15/15 pass (tests/test_config_create_service.py)
• CLI tests: 20/20 pass (tests/test_config_create_cli.py)
• Regression: existing TestConfigNew (5 cases) still passes byte-for-byte
• E2E against real project 1143 (EU GCP) — step 19b dry-run + real create + detail verify + update patch + delete cleanup all green. Live config IDs created and cleaned up across multiple runs.
• Manual smoke: --push without --project → exit 2, VALIDATION_ERROR envelope ✓
• Manual smoke: --push without --name → exit 2 ✓
• --hint client emits a client.create_config(...) snippet; --hint service emits a service.create_config(...) snippet ✓
• Two independent review iterations (code + security) — security verdict CONVERGED with 0 material findings; code review had 5 NON-BLOCKING items, all addressed
• CI green on this PR

Note on E2E pre-existing test failure

The full tests/test_e2e.py::TestFullE2E::test_full_cli_e2e flow currently fails at step 37 (_test_sharing_and_lineage) because the test invokes kbagent lineage show --project P without a required --load argument. This failure is pre-existing — committed on 2026-04-14 by Petr Simecek (line 2499), unrelated to this PR. Step 19b (this PR's new code) passes cleanly in every run; the test harness reaches step 37 only because it's later in the same flow.

🤖 Generated with Claude Code

[padak]

Review of #278 — feat(0.31.1): config new --push for one-shot remote create

Generated by kbagent-pr-reviewer subagent. Verdict and findings below
are advisory; the human author retains every veto. CI-coverable issues
(lint, format, tests) are confirmed via CI check status (green on PR),
not duplicated here.

Summary

This PR extends kbagent config new with a --push flag that POSTs to the Storage
API in one call, filling the F2 gap that previously had no first-class command.
The implementation is architecturally clean, test coverage is thorough (35 new unit
tests, comprehensive flag-combination validation, proper E2E try/finally cleanup),
and all 7 silent-drift surfaces from CLAUDE.md convention #17 are updated.

Verdict: REQUEST CHANGES. Two issues must be fixed before merge:
(1) The PR targets the wrong base — main is now at v0.32.0; the PR's v0.31.1 bump
conflicts, and all (since v0.31.1) version tags ship stale after merge.
(2) When --push --output-dir --json is combined, a plain-text dim line is emitted
to stdout before the JSON object, silently breaking JSON consumers on that flag
combination.

Verdict

• Verdict: REQUEST CHANGES
• Blocking findings: 2
• Non-blocking findings: 3
• Nits: 2

Blocking findings

[B-1] pyproject.toml:4 — Version bump 0.31.1 is obsolete; main is at 0.32.0

The PR increments from 0.31.0 to 0.31.1, but main has merged PR #277 (storage truncate-table, v0.32.0) since this branch diverged. Merging as-is would attempt to
set pyproject.toml version = "0.31.1" against a base that already has
version = "0.32.0", producing a merge conflict (confirmed: mergeStateStatus=DIRTY).
Even if the conflict were manually resolved to keep 0.31.1, it would regress the
already-shipped 0.32.0 entry in changelog.py, plugin.json, and uv.lock.

All 19 occurrences of 0.31.1 in the diff — including gotchas.md's
(since v0.31.1) heading, the matrix entries in keboola-expert.md, and
the scaffold-workflow.md callout — must be re-tagged to the final version
chosen after rebase (0.32.1 for a patch or 0.33.0 for a minor, per the
maintainer's preference). The changelog.py entry must be added under the
new version key, not under "0.31.1".

Fix: rebase onto current main, resolve conflicts, bump pyproject.toml to
0.32.1 (or 0.33.0), run make version-sync, and do a global find-and-replace
of 0.31.1 → chosen version across all doc files.

[B-2] src/keboola_agent_cli/commands/config.py:512 — --push --output-dir --json emits plain text before JSON, breaking stdout parsers

In the --push path when --output-dir is also set, the command calls:

_write_scaffold_to_disk(formatter, scaffold, output_dir, json_mode=False)

json_mode=False is hardcoded regardless of the caller's --json flag.
_write_scaffold_to_disk with json_mode=False calls
formatter.console.print(f"[dim]Scaffold written to ..."), which writes to
formatter.console — the stdout-bound Console instance (not err_console).
formatter.output(push_result) then writes the JSON object to sys.stdout.

Result on stdout with kbagent --json config new ... --push --output-dir ./scratch:

Scaffold written to ./scratch/extractor/.../test-config (1 file(s))
{
  "status": "ok",
  ...
}

The JSON parser sees the first line as non-JSON and fails. The test
test_push_with_output_dir_scaffolds_and_posts (diff line 1269) uses --json
but only asserts result.exit_code == 0 and file presence — it does not call
json.loads(result.output), so the test passes while the bug ships.

Fix: change the call at line 512 to
_write_scaffold_to_disk(formatter, scaffold, output_dir, json_mode=formatter.json_mode),
matching the scaffold-only path at line 530. Also add
json.loads(result.output) assertions to test_push_with_output_dir_scaffolds_and_posts.

Non-blocking findings

[NB-1] plugins/kbagent/agents/keboola-expert.md:80 — config new --push missing from VERSION GATE enumeration

The VERSION GATE block (§1 Rule 6) lists minimum-version requirements line by
line so the AI agent can refuse tasks on old installs. After the rebase the block
ends with project edit --new-alias ... needs 0.31.0+ and storage retype is a future composite. The new config new --push feature (version X.Y.Z after rebase)
is not listed there, even though the PR correctly adds 0.31.1+ annotations in the
Tool Selection Matrix rows and in gotchas.md. An AI agent on a pre-X.Y.Z install
will attempt config new --push based on the matrix entry but not be pre-warned by
the gate check.

CONTRIBUTING.md §Releasing a new version step 5 classifies this as informational
(NON-BLOCKING), but the gap should be closed before the PR ships.

Fix: add one line to the VERSION GATE enumeration after needs 0.31.0+:
\config new --push` (one-shot remote create) needs X.Y.Z+,`
(substituting the final version after rebase).

[NB-2] tests/test_config_create_cli.py:1269 + tests/test_config_create_service.py — No test exercises component_id="keboola.snowflake-transformation" specifically

The PR description and agent docs prominently claim that --push bypasses the MCP
create_config refusal for keboola.snowflake-transformation. This claim is
architecturally correct (the service calls client.create_config directly), but
every service-level test uses keboola.ex-db-snowflake (a different component type).
No unit or CLI test passes component_id="keboola.snowflake-transformation" and
asserts the API POST proceeds. The E2E test (step 19b) uses keboola.ex-http.

If the AI Service schema lookup for keboola.snowflake-transformation behaves
differently from keboola.ex-db-snowflake (e.g. returns a refusal-shaped
error rather than KeboolaApiError), the claim would be unverified at unit-test
level and only caught in production. Adding one parametrized test case with
keboola.snowflake-transformation and configuration=None (the empty-shell path)
would pin the documented behavior.

[NB-3] src/keboola_agent_cli/services/config_service.py:929 — except Exception: in validation fallback is overly broad

_validate_config_body contains:

except Exception:
    logger.warning(...)
    return ("skipped", [])

The comment correctly explains why (late-raising UnknownType from iter_errors),
but a bare except Exception will also silently swallow programming errors such as
AttributeError, TypeError, or MemoryError during the validation loop,
turning them into silent skips. The intent is to catch validation-runtime errors
from jsonschema's lazy evaluation. Narrowing to
except (jsonschema.ValidationError, jsonschema.SchemaError, Exception) is not
better, but catching jsonschema.exceptions.SchemaError plus
jsonschema.exceptions.ValidationError plus an explicit except Exception with a
comment tying it specifically to the known UnknownType failure mode would make
the intent auditable.

Per the severity rubric this is NON-BLOCKING (the intent is documented in the
comment; the behavior is fail-safe rather than fail-dangerous).

Nits

• [NIT-1] src/keboola_agent_cli/commands/config.py:527 — assert scaffold is not None in production code. The comment explains this is a type-narrowing assist, which is fine, but assert statements are suppressed with python -O. Using if scaffold is None: raise RuntimeError(...) would make the guard unconditional without changing the semantics.

• [NIT-2] PR description ends with 🤖 Generated with [Claude Code](https://claude.com/claude-code). Per CONTRIBUTING.md > "Commit & PR Conventions": "No AI attribution footers in PR descriptions." This should be removed before merge.

Verification log

• gh auth status → padak account active, repo scope confirmed ✓
• gh pr view 278 --repo padak/keboola_agent_cli --json state,mergeable,mergeStateStatus → state=OPEN, mergeable=CONFLICTING, mergeStateStatus=DIRTY ✓ (merge conflict confirmed)
• gh pr diff 278 --repo padak/keboola_agent_cli > /tmp/pr278.diff → 2099 lines, 18 files, +1772/-62 ✓
• git log --oneline -3 on origin/main → 8aeeb10 feat(0.32.0): storage truncate-table; pyproject.toml on main = version = "0.32.0" ✓ (B-1 version collision confirmed)
• PR diff pyproject.toml → version = "0.31.1" ✓
• 3-layer check: grep typer/click/formatter/console.print in config_service.py additions → zero hits ✓ (no layer violation in service)
• 3-layer check: grep httpx in commands/config.py additions → zero hits ✓
• grep "config\.new" /tmp/pr278.diff → hints/definitions/config.py registers CommandHint with cli_command="config.new" ✓
• gh api repos/ottomansky/keboola_agent_cli/contents/src%2Fkeboola_agent_cli%2Fpermissions.py?ref=feat%2Fconfig-create → "config.new": "write" at line 42 ✓ (permission entry present)
• VERSION GATE block on fork branch (lines 62-85) → config new --push NOT listed ✓ (NB-1 confirmed)
• commands/config.py diff line 512 → _write_scaffold_to_disk(..., json_mode=False) hardcoded ✓ (B-2 confirmed)
• output.py → self.console = Console(...) (stdout, not stderr) ✓ (B-2 stdout pollution confirmed)
• test_push_with_output_dir_scaffolds_and_posts → asserts exit_code == 0 and written.exists() only; no json.loads(result.output) ✓ (test gap for B-2 confirmed)
• grep "keboola\.snowflake-transformation" in test files → 0 hits in test assertions (only in docs/prose) ✓ (NB-2 confirmed)
• except Exception: in config_service.py:929 → confirmed, correctly guarded with logger.warning ✓ (NB-3 noted)
• TEST_TOKEN = "901-10493007-VDtlEDWDF6Tx5V8jjE8FshFlqM0Hl0c08KHqpt0k" → same synthetic token used across 10+ existing test files (e.g. test_workspace_cli.py:22, test_errors.py:11); not a real credential ✓
• gh pr checks 278 --repo padak/keboola_agent_cli → check pass 1m44s ✓ (CI green on the PR's own branch; make check on working tree fails only due to unrelated 0.32.0 changelog gap on the local branch)
• E2E step 19b try/finally: new_config_id captured before assertions (diff line 2032); registered in _created_config_ids before try: block; finally: runs config delete ✓ (teardown ordering correct)
• Snowflake bypass claim: client.create_config exists on main at client.py:691; service calls it without any component-type filter ✓ (claim is architecturally correct; unit test coverage gap NB-2 noted)
• Behavior verification: not run against real API due to cross-fork PR and missing live credentials; E2E author-confirmed against project 1143 per PR description.

Open questions for the author

(none)

[padak]

Hi @ottomansky — heads up: this PR is being consolidated with #279 into a single 0.33.0 joint release at #282.

What happened:

• feat(0.31.1): config new --push for one-shot remote create #278 (this PR) and fix(0.31.1): data-app --json emits app_id (renamed from id) for --app-id symmetry #279 both targeted a base commit before feat(0.32.0): kbagent storage truncate-table — row-level truncation that preserves schema #277 (storage truncate-table, v0.32.0) merged. That made the original 0.31.1 version label obsolete and left both branches in a CONFLICTING state.
• Rather than ask you to rebase each one independently, we rebased both onto current main (post-feat(0.32.0): kbagent storage truncate-table — row-level truncation that preserves schema #277), cherry-picked your two commits (authorship preserved verbatim — see the Author: ottomansky lines in feat(0.33.0): config new --push + data-app app_id rename (joint release of #278 + #279) #282), re-tagged everything from 0.31.1 to 0.33.0 (12 string replacements across the docs), and applied the one follow-up fix from this PR's first review pass:
  • B-2 (commands/config.py --push --output-dir path): the previous fix attempt swapped json_mode=False for json_mode=formatter.json_mode, which closed the human-mode dim-banner leak but emitted two concatenated JSON documents in JSON mode (the scaffold envelope and the push-result envelope) — broke jq pipes. The clean fix is a new silent=True parameter on _write_scaffold_to_disk that suppresses both the dim banner and the side-channel JSON envelope when the caller already owns the authoritative output. Regression test now asserts json.loads(result.output) parses cleanly.

The merge-conflict resolution kept BOTH new VERSION GATE entries in keboola-expert.md (your data-app * line from #279 sits next to the storage truncate-table line from #277, plus a new config new --push line for this PR's contribution).

Plan: once #282 merges, this PR will be closed as superseded. Your Git authorship is preserved on both feature commits in #282; the release notes for 0.33.0 explicitly credit you for the implementation, live-API validation, and the 7-surface doc sync.

Thanks for the work on this — --push is a great FIIA migration unblock, especially the Snowflake-transformation matrix fix.

[padak]

Superseded by #282 (joint 0.33.0 release). See comment above for context.