Skip to content

feat(0.26.0): config set-default-bucket — discoverable storage.output.default_bucket override - #234

Merged
padak merged 1 commit into
keboola:mainfrom
ottomansky:feat/config-set-default-bucket
May 1, 2026
Merged

feat(0.26.0): config set-default-bucket — discoverable storage.output.default_bucket override#234
padak merged 1 commit into
keboola:mainfrom
ottomansky:feat/config-set-default-bucket

Conversation

@ottomansky

Copy link
Copy Markdown
Contributor

Summary

Adds kbagent config set-default-bucket --project P --component-id C --config-id ID (--bucket BUCKET_ID | --clear) [--dry-run] [--branch ID] — a first-class command for setting or clearing configuration.storage.output.default_bucket without raw-mode JSON edits.

Surfaces the workaround documented at https://keboola.atlassian.net/wiki/spaces/SUP/pages/3770155030/ (epic KBCP-108 "Default bucket is not mandatory") as a discoverable command — visible in --help, kbagent context, the SKILL decision table, and CLAUDE.md.

  • Read-modify-write preserves all sibling keys under storage.output and the rest of the configuration
  • --dry-run shows the diff without writing (same shape as config update --dry-run)
  • Same-value writes short-circuit with {"changed": false} — no API call, no version bump
  • --clear removes the key, leaves storage.output: {} if no siblings (intentional, mirrors set_nested_value parent-creation semantics)

kbagent config update --set 'storage.output.default_bucket=...' already worked, but it isn't discoverable. The new wrapper is what shows up when humans or AI agents look for "how do I change the default bucket".

Live validation against a real project

Tested end-to-end against three different component types in throwaway dev branches — output tables routed to the configured bucket at job runtime in every case:

Component Result
kds-team.ex-google-cloud-storage (existing, row-based) ✅ output tables landed in the configured bucket
keboola.ex-cnb-exchange-rates (new config, root-only) cnb-rates table landed in the configured bucket
ex-generic-v2 (new config, REST→tables, multi-job) ✅ both users AND posts tables landed in the same configured bucket

Also validated: per-table destination override (the second method shown in the support article) works through existing kbagent config update --set 'storage.output.tables=[...]' — no new wrapper added there because per-table mappings have many fields and a single-purpose flag would be cramped.

All test branches and configs were cleaned up after validation.

Changes

New code

  • services/config_service.py: ConfigService.set_default_bucket() — read-modify-write with validation, no-op detection, dry-run shape mirroring update_config()
  • commands/config.py: config set-default-bucket Typer command with --bucket / --clear (mutually exclusive), --dry-run, --branch, hint short-circuit, dual JSON/human output
  • hints/definitions/config.py: CommandHint registered with ClientCall + ServiceCall (both --hint client and --hint service produce valid Python)
  • permissions.py: "config.set-default-bucket": "write"

Tests

  • tests/test_config_set_default_bucket.py (new): 20 service + CLI tests covering set/clear/no-op/dry-run/sibling-preservation/validation/branch propagation/client lifecycle
  • tests/test_e2e.py: step 18c added — set → detail → no-op → clear roundtrip against a real config

Docs

  • commands/context.py — entry in AGENT_CONTEXT
  • CLAUDE.md — line in ## All CLI Commands
  • plugins/kbagent/skills/kbagent/SKILL.md — auto-regenerated
  • plugins/kbagent/skills/kbagent/references/commands-reference.md — Configuration Browsing section
  • plugins/kbagent/skills/kbagent/references/gotchas.md — note about default_bucket semantics + KBCP-108 link
  • plugins/kbagent/skills/kbagent/references/safe-write-workflow.md — listed as a safe pattern alongside config update --set
  • docs/e2e-scenarios.md — step 18c row added (also caught a missing 18b row for config rename)
  • changelog.py0.22.0 entry

Version

  • pyproject.toml: 0.21.10.22.0
  • plugins/kbagent/.claude-plugin/plugin.json: synced via make version-sync
  • uv.lock: package version updated

Test plan

  • make lint — clean
  • make format-check — clean
  • make test — 1871 passed, 4 skipped, 34 deselected
  • make skill-check — clean (SKILL.md regenerated and matches)
  • make version-check — clean (plugin.json at 0.22.0)
  • make check (combined) — clean
  • Live E2E validated on three component types (see table above)
  • kbagent --hint client config set-default-bucket ... — produces valid Python with the correct stack URL
  • kbagent --hint service config set-default-bucket ... — produces valid Python with proper service args (incl. --clear correctly omitting bucket)
  • CI run on PR

@ottomansky
ottomansky force-pushed the feat/config-set-default-bucket branch from cf2cb1a to dd2a0b5 Compare April 30, 2026 19:29
@ottomansky ottomansky changed the title feat(0.22.0): config set-default-bucket — discoverable storage.output.default_bucket override feat(0.26.0): config set-default-bucket — discoverable storage.output.default_bucket override Apr 30, 2026
@ottomansky

Copy link
Copy Markdown
Contributor Author

Rebased onto upstream/main (now at 0.25.3). Conflicts resolved:

  • Version: bumped to 0.26.0 (next minor, since this is a feature on top of 0.25.3).
  • changelog.py: prepended a 0.26.0 entry above 0.25.3.
  • gotchas.md: my entry now tagged (since 0.26.0) and placed at the top.
  • hints/definitions/config.py: kept all of upstream's new hint blocks (config.metadata-list, config.get-metadata, config.set-metadata, config.delete-metadata, config.set-folder); my new config.set-default-bucket hint appended at the end.
  • uv.lock + plugins/kbagent/.claude-plugin/plugin.json: regenerated via uv sync and make version-sync against the new pyproject version.

Also picked up new mandatory surfaces I missed in the original PR:

  • Migrated raw error_code string literals to ErrorCode enumscripts/check_error_codes.py (introduced in 0.24.0) was rejecting my code; now uses ErrorCode.VALIDATION_ERROR / ErrorCode.CONFIG_ERROR.
  • plugins/kbagent/agents/keboola-expert.md tool-selection matrix: added a row for the new command (highest silent-drift surface per CLAUDE.md convention v0.6.0: Branch lifecycle management + security hardening #17).

make check passes locally — 2351 tests, lint clean, format clean, skill fresh, version synced, error-code script clean. PR is mergeable.

@ottomansky
ottomansky force-pushed the feat/config-set-default-bucket branch from dd2a0b5 to 75a24b5 Compare April 30, 2026 19:43
@ottomansky

Copy link
Copy Markdown
Contributor Author

Self-review pass complete (per CONTRIBUTING.md ## Self-review before tagging a human). Fixes pushed in 75a24b5:

Blocking (fixed):

  1. plugins/kbagent/agents/keboola-expert.md §3 inline gotcha — my new bullet incorrectly claimed dev branches universally prefix realized buckets with the branch ID. That's only the runner's behavior on legacy fake-branch projects (already covered by the 0.25.2 legacy_branch_storage gotcha). Reworded to point at the existing 0.25.2 entry instead of duplicating the claim.
  2. src/keboola_agent_cli/services/config_service.py set pathset_nested_value("storage.output.default_bucket", ...) would raise an unstructured KeyError if the existing config had storage = None or storage.output = None (raw-mode JSON edits can serialize nulls). Replaced with semantic no-op detection (compare existing to target bucket/None BEFORE mutating) followed by a normalize-then-write that tolerates None parents. Added two tests for storage=None (set + clear) and one for output=None.

Also picked up while here:

  • SKILL.md description triggers — added default bucket, output bucket, default_bucket, storage.output, raw mode bucket override, custom output bucket name so AI agents can find this skill via the article's keywords.
  • keboola-expert.md Rule 6 VERSION GATE — added config set-default-bucket needs 0.26.0+ to the version-gate examples list.
  • keboola-expert.md §3 Inline Gotchas — added the (0.26.0+) bullet (with the corrected scope from blocker 1).

make check clean — 2354 tests (3 new edge-case tests), lint clean, format clean, skill fresh, version synced, error-code script clean.

Reviewed against the full Plugin synchronization map and kbagent-pr-reviewer step 3.4 rubric — every silent-drift surface has a corresponding line in the diff.

@ottomansky
ottomansky force-pushed the feat/config-set-default-bucket branch from 75a24b5 to fe7ea8b Compare April 30, 2026 19:53
…t.default_bucket override

Add a first-class CLI command for setting/clearing
configuration.storage.output.default_bucket without raw-mode JSON edits.
Surfaces the workaround documented in the support article (linked from
epic KBCP-108) as a discoverable command in --help, kbagent context, and
the SKILL decision table. Read-modify-write that preserves all sibling
keys; --dry-run for preview; same-value writes short-circuit with
{"changed": false}.

- Service: ConfigService.set_default_bucket() with validation, no-op
  detection, dry-run shape mirroring update_config().
- Command: config set-default-bucket with --bucket/--clear (mutually
  exclusive), --dry-run, --branch, hint short-circuit.
- Hint registered (client + service variants) in
  hints/definitions/config.py.
- Permission "config.set-default-bucket": "write".
- Tests: 20 service+CLI unit tests + step 18c in test_e2e.py covering
  set/dry-run/clear/no-op against a real config.
- Docs: AGENT_CONTEXT, CLAUDE.md All CLI Commands, commands-reference,
  gotchas (tagged since 0.26.0), safe-write-workflow,
  keboola-expert.md tool selection matrix, docs/e2e-scenarios.
- Live-validated end-to-end on three component types (GCS extractor /
  CNB exchange rates / Generic extractor) -- output tables routed to
  the configured bucket at job runtime in every case.
@ottomansky
ottomansky force-pushed the feat/config-set-default-bucket branch from fe7ea8b to 3ec6f34 Compare April 30, 2026 19:58
@ottomansky

Copy link
Copy Markdown
Contributor Author

Review loop complete (head 3ec6f34).

Iteration 1 (self-walk against Plugin synchronization map): added every silent-drift surface, version-synced, regenerated SKILL.md, added e2e step.

Iteration 2 (independent review): found 2 BLOCKING + 3 NON-BLOCKING + 3 nits. Fixed all:

  • B-1: keboola-expert.md §3 inline gotcha incorrectly claimed <branch_id>- prefix is universal -- it's only on legacy fake-branch projects. Reworded.
  • B-2: Service set_nested_value path would crash unstructured on storage=None / storage.output=None. Replaced with semantic no-op detection + normalize-then-write. Added 3 tolerance tests.
  • N-1: Service-layer validation pattern matches existing update_config (kept for consistency).
  • N-2/N-3 (citation/PII): not applicable.
  • nits: README cheat sheet updated with set-default-bucket and set-folder, cosmetic target fold, duplicate assert_not_called removed, JSON-string-configuration test added.

Iteration 3 (independent review): 0 BLOCKING + 0 NON-BLOCKING + 1 material nit. Fixed:

  • e2e step 18c claimed sibling-key preservation but didn't actually seed a sibling. Test now seeds storage.output.tables=[] first and asserts the sibling survives both --bucket and --clear.

Iteration 4 (sign-off): zero findings. make check clean, 2355 tests pass, no debug artifacts, no token leaks, all silent-drift surfaces consistent.

Loop terminated. PR is mergeable; CI re-running on 3ec6f34.

@padak
padak merged commit 56ad19b into keboola:main May 1, 2026
1 check passed
@ottomansky
ottomansky deleted the feat/config-set-default-bucket branch May 11, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants