Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"plugins": [
{
"name": "kbagent",
"version": "0.88.0",
"version": "0.89.0",
"source": "./plugins/kbagent",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, sync configs as files, manage dev branches, and debug SQL in workspaces",
"category": "development"
Expand Down
11 changes: 11 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -377,6 +377,17 @@ kbagent config search --query PATTERN [--project NAME] [--component-type TYPE] [
kbagent config update --project NAME --component-id ID --config-id ID [--name N] [--description D] [--configuration JSON|@file|-] [--configuration-file PATH] [--set PATH=VALUE ...] [--merge] [--change-description TEXT] [--dry-run] [--branch ID] [--allow-plaintext-on-encrypt-failure]
kbagent config set-default-bucket --project NAME --component-id ID --config-id ID (--bucket BUCKET_ID | --clear) [--dry-run] [--branch ID]
kbagent config rename --project NAME --component-id ID --config-id ID --name "New Name" [--branch ID] [--directory DIR]
kbagent config delete --project NAME --component-id ID --config-id ID [--branch ID] [--dry-run]
kbagent config restore --project NAME --component-id ID --config-id ID [--branch ID]
kbagent config trash-list --project NAME [--component-id ID] [--branch ID]
# config delete (0.89.0+ safety): SOFT delete into the Storage trash, with a locate-first guard.
# The raw API purges PERMANENTLY when DELETE hits a config already in the trash -- the classic
# agent retry after a timeout. kbagent now looks the config up first: live -> trash it;
# already trashed -> status "already_in_trash", exit 0, NO second DELETE ever sent; absent
# from both -> NOT_FOUND. Undo with `config restore`; browse candidates with `config
# trash-list`. Before 0.89.0 the second delete destroyed the config permanently. CLAUDE.md
# did not list `config delete` at all until 0.89.0 (silent drift) -- the command itself has
# existed for a long time.
kbagent config variables-set --project NAME --component-id ID --config-id ID --var KEY=VALUE [--var ...] [--replace] [--variables-id ID] [--values-id ID] [--branch ID] [--dry-run]
kbagent config variables-get --project NAME --component-id ID --config-id ID [--branch ID]
kbagent config variables-clear --project NAME --component-id ID --config-id ID [--branch ID] [--yes]
Expand Down
2 changes: 1 addition & 1 deletion plugins/kbagent/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "kbagent",
"version": "0.88.0",
"version": "0.89.0",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, sync configs as files, manage dev branches, and debug SQL in workspaces",
"author": {
"name": "Keboola",
Expand Down
4 changes: 3 additions & 1 deletion plugins/kbagent/skills/kbagent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ When working inside a git repository or project directory, run `kbagent init` (o
| Update a configuration's metadata and/or content | `kbagent config update --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| Set or clear ``storage.output.default_bucket`` on a configuration | `kbagent config set-default-bucket --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| Rename a configuration (update name via API + rename local sync directory) | `kbagent config rename --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --name NAME` |
| Delete a configuration from a project | `kbagent config delete --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| Soft-delete a configuration into the trash (restorable) | `kbagent config delete --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| Generate boilerplate configuration files for a Keboola component, optionally creating the config remotely in one shot | `kbagent config new --component-id COMPONENT-ID` |
| List all metadata entries on a configuration | `kbagent config metadata-list --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| Read a single metadata value by key | `kbagent config get-metadata --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --key KEY` |
Expand All @@ -130,6 +130,8 @@ When working inside a git repository or project directory, run `kbagent init` (o
| Overwrite the runtime ``state`` dict of a configuration or one of its rows | `kbagent config state-set --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --state STATE` |
| Duplicate a configuration, whole -- including runtime, storage and authorization | `kbagent config clone --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID --name NAME` |
| Requires master token. | `kbagent config oauth-url --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| Restore a configuration from the trash (undo of 'config delete') | `kbagent config restore --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
| List configurations in the trash (restorable via 'config restore') | `kbagent config trash-list --project PROJECT` |
| List data apps across one or more registered projects | `kbagent data-app list` |
| Show merged Data Science + Storage detail for one data app | `kbagent data-app detail --project PROJECT --app-id APP-ID` |
| Create a Keboola data app end-to-end (POST + encrypt + PUT + deploy) | `kbagent data-app create --project PROJECT --name NAME --slug SLUG` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,9 @@ Requires a **super-admin** Manage API token (same kind as `org setup`). Same def
- `config update --project NAME --component-id ID --config-id ID [--name N] [--description D] [--configuration JSON|@file|-] [--configuration-file PATH] [--set PATH=VALUE ...] [--merge] [--change-description TEXT] [--dry-run] [--branch ID] [--allow-plaintext-on-encrypt-failure]` -- update metadata and/or configuration content. `--change-description` sets the new config version's `changeDescription` (the version-history audit line); omit it to keep the auto-generated default (e.g. `Updated configuration via kbagent config update`). Distinct from `--description`, which sets the config's display description. `--dry-run` echoes the `change_description` that would be sent. **`#`-prefixed secrets auto-encrypt via the Encryption API before write (fail-closed; since 0.54.0, #378)** -- `--allow-plaintext-on-encrypt-failure` overrides, `--dry-run` keeps plaintext in the diff (ciphertext is non-deterministic). Note `--set '#password=...'` sets a *top-level* key; for a nested secret use `--set 'parameters.#password=...'`. `--set` targets a nested key (e.g. `parameters.db.host=new-host`). `--merge` deep-merges into existing config (preserves sibling keys). `--dry-run` previews changes without applying. Paths are relative to the configuration root. **Auto-normalize (0.28.0+; #245 / 0.31.0+; #274)**: `parameters.blocks[].codes[].script` is fixed before pushing to Storage API. **String -> array** (0.28.0+; #245): SQL transformations get statement-level split (respects `'...'` / `"..."` / `$$..$$` / `--` / `#` / `//` / `/* ... */`); Python / R / `kds-team.app-custom-python` get `[script]` wrap. **List-element re-split** (0.31.0+; #274): when `script` is already a list but an element packs multiple `;`-separated statements, each SQL element is re-run through `split_statements()` and replaced inline. Closes the ODBC `Actual statement count N did not match the desired statement count 1` (SQL state 0A000) runtime crash that survives the 0.28.0 string fix. The result envelope's `normalizations: [{path, action: "sql_split"|"wrap_array"|"sql_resplit", before_type, after_type, after_length, before_length?}]` records every change (empty when nothing was malformed; `sql_resplit` adds `before_length` and a `[E]` suffix on `path` pointing at the original element index). Bypassing kbagent (raw REST) does NOT inherit either pass -- prefer `kbagent config update` for SQL transformation body changes.
- `config set-default-bucket --project NAME --component-id ID --config-id ID (--bucket BUCKET_ID | --clear) [--dry-run] [--branch ID]` -- set or clear `configuration.storage.output.default_bucket` on a configuration. Discoverable shortcut for the raw-mode workaround at https://keboola.atlassian.net/wiki/spaces/SUP/pages/3770155030/. Read-modify-write that preserves sibling keys; returns `{"changed": false}` when the value already matches the requested state. Honored by output tables that don't pin their own `destination`.
- `config rename --project NAME --component-id ID --config-id ID --name "New Name" [--branch ID] [--directory DIR]` -- rename a configuration (API update + local sync directory rename with git mv support)
- `config delete --project NAME --component-id ID --config-id ID [--branch ID]` -- delete a configuration
- `config delete --project NAME --component-id ID --config-id ID [--branch ID] [--dry-run]` -- SOFT-delete a configuration into the Storage trash (restorable). Since 0.89.0 it locates the config first and a config already in the trash is NOT deleted again -- the raw API purges permanently on a second DELETE (the retry-after-timeout trap); kbagent reports `already_in_trash` and exits 0 instead. `--dry-run` reports the located state without writing. Undo with `config restore`. Permission class `destructive`
- `config restore --project NAME --component-id ID --config-id ID [--branch ID]` *(since v0.89.0)* -- restore a trashed configuration (versions, rows and metadata come back). Only works on a config currently in the trash. Permission class `write`
- `config trash-list --project NAME [--component-id ID] [--branch ID]` *(since v0.89.0)* -- list configurations in the trash; each row carries `component_id`, `config_id`, `name`, `version` and `deleted_at`, which is exactly what `config restore` needs. Permission class `read`
- `config new --component-id ID [--project NAME] [--name NAME] [--output-dir DIR] [--push --no-files --description D --configuration JSON|@file|- --configuration-file PATH --no-validate --branch ID --dry-run --allow-plaintext-on-encrypt-failure]` -- **two modes**. **Default (no `--push`)**: scaffold new config from component schema; writes files to `--output-dir` or prints to stdout. **Zero API calls.** **With `--push`** (0.33.0+, requires `--project` + non-empty `--name`): also POSTs to `/v2/storage/components/{cid}/configs` for a one-shot remote create. `#`-prefixed secrets in the pushed body auto-encrypt via the Encryption API first (fail-closed; since 0.54.0, #378; `--allow-plaintext-on-encrypt-failure` overrides). `--no-files` skips the filesystem step entirely (FIIA-style empty-shell pattern). `--configuration` / `--configuration-file` override the POSTed body (default is `{}`, with validation auto-skipped for the default empty shell). `--dry-run` previews the planned POST + validation result without creating. Schema validation runs by default when an explicit body is given (fail-closed: `ConfigError` exit 5 on mismatch) but skips silently if the AI Service has no schema for the component or returns an error; `--no-validate` opts out. Works for ALL component types including `keboola.snowflake-transformation`.
- `config clone --project P --component-id ID --config-id ID --name NAME [--target-project P2] [--description D] [--set PATH=VALUE ...] [--secret PATH=VALUE ...] [--branch ID] [--target-branch ID] [--dry-run] [--allow-plaintext-on-encrypt-failure]` (0.84.2+, #587) -- duplicate a configuration **whole**. Reach for this instead of reading `config detail` and rebuilding a body: copying only `configuration["parameters"]` silently drops its siblings (`runtime`, `storage`, `authorization`), and a lost `runtime.parallelism` makes Keboola fall back to `parallelism: 1` -- the reporter's 65-row writer went sequential, 140 min instead of ~60-90, with nothing in any output pointing at it. **Same project** (default): server-side copy via `POST .../configs/{id}/versions/{v}/create`; rows and `KBC::` encrypted values travel with it (verified live). `--set PATH=VALUE` is applied as a follow-up update on the copy, so an override can never be the reason a key went missing. **Cross project** (`--target-project`): reassembled client-side and rows recreated one by one, because encrypted values **cannot** travel -- a Keboola ciphertext is scoped to the project it was encrypted in. Any `KBC::` value makes the clone **fail with exit 5**, listing every path, until re-supplied via `--secret PATH=VALUE` (encrypted in the TARGET project on write). `--dry-run` reports those paths instead of refusing -- run it first to learn what to gather. Storage bucket/table IDs are copied **verbatim, never remapped**; `sync clone` is the command that remaps.
- `config variables-set --project NAME --component-id ID --config-id ID --var KEY=VALUE [--var ...] [--replace] [--variables-id ID] [--values-id ID] [--branch ID] [--dry-run] [--allow-plaintext-on-encrypt-failure] [--yes]` -- attach variable values to a config. Auto-creates a sibling `keboola.variables` config + default row on first use and links it via the parent's `runtime.variables_id` / `variables_values_id`. Defaults to merge; `--replace` drops keys not in `--var`. `#`-prefixed values encrypt via the Encryption API (fail-closed; exit non-zero on `ENCRYPTION_FAILED`). See `variables-workflow.md`
Expand Down
32 changes: 32 additions & 0 deletions plugins/kbagent/skills/kbagent/references/gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -1327,6 +1327,38 @@ events and emits a final `done` SSE frame mirroring the same record.
error names the created configuration id and how many rows landed, so you
can delete it and re-run.

## `config delete` twice = permanent purge -- kbagent now refuses the second one (since v0.89.0)

The Storage API overloads `DELETE .../configs/{id}`: on a live configuration
it soft-deletes into the trash (restorable), but on a configuration ALREADY in
the trash the same call **purges it permanently** -- versions, rows and
metadata gone, no restore. The classic way to hit this is an agent retry: the
first DELETE times out client-side after the server already trashed the
config, the retry fires, and the retry destroys it for good.

- Since 0.89.0 `config delete` locates the configuration first and never
sends a DELETE at anything that is not live. A config already in the trash
answers `status: "already_in_trash"` with **exit 0** (the retry stays
idempotent for scripts) and a pointer to `config restore`. A config in
neither place is NOT_FOUND.
- `config restore --project P --component-id C --config-id ID` is the undo;
`config trash-list` shows what is restorable. Restore brings back versions,
rows and metadata.
- **On kbagent <= 0.88.x the guard does not exist** -- a repeated
`config delete` there purges permanently. When driving an older kbagent,
never blind-retry a delete; check `config list` first.
- Direct API callers: the purge-safe alternative is the dedicated
`POST .../configs/{id}/purge` endpoint (fails with 400 when the config is
not in the trash), never a second DELETE.
- **The likeliest second DELETE is not a human retry -- it is the HTTP
client's own.** `DELETE` is conventionally idempotent, so most transports
(kbagent's included, before 0.89.0) repeat it after a read timeout or a
5xx. On this endpoint that automatic repeat IS the purge, and it happens
before any caller sees a result. If you are writing a script or another
client against the Storage API, disable transport-level retry for a config
DELETE specifically; idempotency here is a property of the endpoint, not
of the method.

## `data-app` JSON output: key for the app's own id is `app_id` (since v0.33.0)

- Every `kbagent --json data-app <subcommand>` envelope emits the
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "keboola-cli"
version = "0.88.0"
version = "0.89.0"
description = "AI-friendly CLI for managing Keboola projects"
readme = "README.md"
requires-python = ">=3.12"
Expand Down
24 changes: 24 additions & 0 deletions src/keboola_agent_cli/changelog.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,30 @@

# Ordered newest-first. Each value is a list of brief one-line descriptions.
CHANGELOG: dict[str, list[str]] = {
"0.89.0": [
"Fix: `config delete` can no longer permanently purge a configuration by being run "
"twice. The Storage API overloads DELETE -- on a live configuration it soft-deletes "
"into the trash, but on a configuration ALREADY in the trash the same call purges it "
"permanently, versions, rows and metadata included. A timed-out delete followed by a "
"retry is exactly that second call, and retrying on timeout is what every agent and CI "
"script does. `config delete` now locates the configuration first and never sends a "
"DELETE at anything that is not live: already trashed answers `already_in_trash` with "
"exit 0 (the retry stays idempotent), absent from both answers NOT_FOUND. `--dry-run` "
"reports the located state without writing.",
"Fix: the config DELETE is no longer retried by the HTTP layer itself. `DELETE` sits in `RETRY_SAFE_METHODS`, so a read timeout or a 5xx made the transport repeat it automatically -- and for THIS endpoint the repeat is the purge. The server trashes the config, the response is lost, the retry lands on the now-trashed config and destroys it before any caller sees a result. The service-level locate-first guard runs once per call and cannot see inside that loop, so `client.delete_config` now passes a new per-call `retry_safe=False` override and a lost response surfaces as a TIMEOUT the caller decides about; re-running the command is safe because the guard catches the trashed state. Every other DELETE keeps the retry -- the opt-out is per endpoint, because idempotency is a property of the endpoint, not of the method.",
"Note: `locate_config` no longer infers a live state from the absence of a 404 either. Every stack checked answers 404 for a trashed configuration (verified live on connection.keboola.com and the GCP stack), but a body carrying `isDeleted: true` is now read as trashed regardless of status code -- that flag decides whether a purge-capable DELETE goes out, so it is read rather than inferred.",
"New: `kbagent config restore` -- the undo for `config delete`. Restores a trashed "
"configuration with its versions, rows and metadata (`POST .../configs/{id}/restore`). "
"Only works on a configuration currently in the trash.",
"New: `kbagent config trash-list` lists configurations in the trash, project-wide or "
"narrowed by `--component-id`. Each row carries component_id, config_id, name, version "
"and deleted_at -- exactly what `config restore` needs.",
"Note: all three are mirrored on `kbagent serve`. `DELETE /configs/...` gains "
"`dry_run`; `POST /configs/{p}/{c}/{id}/restore` and `GET /configs/trash/{p}` are new.",
"Plugin docs: `CLAUDE.md`'s command list had never included `config delete` at all. "
"That silent drift made the command look nonexistent to AI agents reading it. "
"Added alongside the new commands, with the double-delete trap recorded in gotchas.md.",
],
"0.88.0": [
"Fix (#624): column descriptions are now written where the Keboola UI and the "
"MCP server actually read them. Until 0.87.0 `storage describe-column` / "
Expand Down
12 changes: 9 additions & 3 deletions src/keboola_agent_cli/client/_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,15 @@ def __enter__(self) -> Self:
def __exit__(self, *args: Any) -> None:
self.close()

def _request(self, method: str, path: str, **kwargs: Any) -> httpx.Response:
"""Execute a Storage API request with retry."""
return self._do_request(method, path, **kwargs)
def _request(
self, method: str, path: str, retry_safe: bool | None = None, **kwargs: Any
) -> httpx.Response:
"""Execute a Storage API request with retry.

``retry_safe=False`` opts a single call out of the method-based retry
rule -- see :meth:`BaseHttpClient._do_request`.
"""
return self._do_request(method, path, retry_safe=retry_safe, **kwargs)

def _get_or_create_sub_client(
self,
Expand Down
Loading