From 0328514274e5e9611b77f29caa6ce21b2f6765bd Mon Sep 17 00:00:00 2001 From: Petr Date: Tue, 25 Aug 2026 15:12:34 +0200 Subject: [PATCH 1/2] docs(plugin): polish workspace-load skill guidance after #692 - SKILL.md triggers: zero-copy clone, workspace load type, table too big to load, load table into workspace cheaply - workspace-workflow: cheapest-first ordering (skip load > clone > view > copy), verify-via-tables[] note, and fix a misleading example -- the size guard only fires for resolved COPY, so --force on an explicit clone is a no-op; --json mode never prompts (--force up front) - typify-table / storage-types workflows: warn that a large typeless table COPY needs --force in non-interactive mode - keboola-expert: STORAGE_JOB_TIMEOUT is now exit 4 (retryable) and the job keeps running server-side --- plugins/kbagent/agents/keboola-expert.md | 2 +- plugins/kbagent/skills/kbagent/SKILL.md | 3 +- .../references/storage-types-workflow.md | 3 ++ .../references/typify-table-workflow.md | 7 ++- .../kbagent/references/workspace-workflow.md | 44 ++++++++++++++----- 5 files changed, 46 insertions(+), 13 deletions(-) diff --git a/plugins/kbagent/agents/keboola-expert.md b/plugins/kbagent/agents/keboola-expert.md index d4052c2f..3b419966 100644 --- a/plugins/kbagent/agents/keboola-expert.md +++ b/plugins/kbagent/agents/keboola-expert.md @@ -124,7 +124,7 @@ been retired, so its absence is NOT a promise (see ยง1 Rule 6). | Re-seed a table without losing schema / PK / dependents | `kbagent storage truncate-table --project P --table-id in.c-foo.data [--dry-run] [--yes]` -- rows only, uniformly async-via-job on every branch; batch via repeated `--table-id` | -- | drop + recreate (loses descriptions, PK, sharing edges, and breaks every downstream reference); deleting rows via raw SQL in a workspace (bypasses the Storage audit trail) | | Back up / restore a table around a risky change | `kbagent storage snapshot-create --table-id ...` then, to restore, `kbagent storage table-from-snapshot --snapshot-id ID --bucket-id B --name NEW` -- restore is always a NEW table (`--name` REQUIRED, no overwrite): verify it, then `swap-tables`. See [snapshot-workflow.md](../skills/kbagent/references/snapshot-workflow.md) | `storage snapshots` / `snapshot-detail` to find one | exporting to CSV as a "backup" (loses column types + PK); `create-table --snapshot-id` (not a thing) | | Debug a failed job | `kbagent job detail --project P --job-id J --json` + `kbagent job run ... --log-tail-lines 200` | `kbagent workspace from-transformation` for SQL repro | "I think the issue is..." without reading logs | -| Ad-hoc SQL / row-count / type audit | `kbagent workspace create` + `workspace load` (since vNEXT auto-CLONEs eligible tables, else COPY; `--load-type` forces one and fails loudly if ineligible; COPY > 1 GiB needs `--force` outside a TTY) + `kbagent workspace query --sql "..."` -- results are inline and fast but **capped at `--limit`, default 500**: check `statements[].truncated` / `total_rows`, use `COUNT(*)` for counts, `--full` for the complete set | `kbagent workspace from-transformation` for existing-transform debugging; `workspace list --qs-compatible` for data-app reuse; read-only input-mapping (`KBC__`) to query prod with no load at all | trusting a default `SELECT *` as the full result; querying Storage via raw Snowflake credentials outside the workspace abstraction | +| Ad-hoc SQL / row-count / type audit | `kbagent workspace create` + `workspace load` (since vNEXT auto-CLONEs eligible tables, else COPY; `--load-type` forces one and fails loudly if ineligible; COPY > 1 GiB needs `--force` outside a TTY; on a `--timeout` [default 300s] the job keeps running server-side and now exits 4/retryable, not 1 -- retry or poll `GET /v2/storage/jobs/{id}`, don't treat it as a hard failure) + `kbagent workspace query --sql "..."` -- results are inline and fast but **capped at `--limit`, default 500**: check `statements[].truncated` / `total_rows`, use `COUNT(*)` for counts, `--full` for the complete set | `kbagent workspace from-transformation` for existing-transform debugging; `workspace list --qs-compatible` for data-app reuse; read-only input-mapping (`KBC__`) to query prod with no load at all | trusting a default `SELECT *` as the full result; querying Storage via raw Snowflake credentials outside the workspace abstraction | | Export a FILTERED or INCREMENTAL slice of a table (no workspace) | `kbagent storage download-table --table-id ... --where-column status --where-value active [--where-operator eq\|neq] [--changed-since "-2 days"]` -- server-side filter on the credential-only export path | `kbagent workspace query` with a `WHERE` clause when you need real SQL | downloading the whole table then filtering locally | | Run Keboola SQL / read-write Storage Files from INSIDE a Python process you control | `from keboola_agent_cli import Client` -- stateless `Client(url, token)`; `.query(workspace_id, sql)`, `.files.upload/.read_bytes/.list`; no subprocess, no `serve`, no config-dir. See [library-workflow.md](../skills/kbagent/references/library-workflow.md) | the CLI or `kbagent serve` REST when you are NOT already inside Python | shelling out to the `kbagent` binary from Python you control; using it for open-ended exploration (fixed set of typed ops) | | Inspect dev branch | `kbagent branch list --project P`, `kbagent branch use --project P --branch ID` | -- | acting on `main` when a dev branch exists | diff --git a/plugins/kbagent/skills/kbagent/SKILL.md b/plugins/kbagent/skills/kbagent/SKILL.md index 2bac3171..9ef95191 100644 --- a/plugins/kbagent/skills/kbagent/SKILL.md +++ b/plugins/kbagent/skills/kbagent/SKILL.md @@ -16,7 +16,8 @@ description: > feature flag, flow schedule, invite member, SQL transformation edit, sync action, keboola docs, table snapshot, auth, login, sign in, PAYG credits, flow notifications, alert recipients, config trash, - restore config, undelete config. + restore config, undelete config, zero-copy clone, workspace load type, + table too big to load, load table into workspace cheaply. --- # kbagent -- Keboola Agent CLI diff --git a/plugins/kbagent/skills/kbagent/references/storage-types-workflow.md b/plugins/kbagent/skills/kbagent/references/storage-types-workflow.md index 1a2a9a96..7fff28c8 100644 --- a/plugins/kbagent/skills/kbagent/references/storage-types-workflow.md +++ b/plugins/kbagent/skills/kbagent/references/storage-types-workflow.md @@ -246,6 +246,9 @@ kbagent branch use --project prod --branch # 2. In a workspace, build a typed CTAS into a sibling table kbagent workspace create --project prod +# Default auto-picks clone (fast, any size) when eligible, else copy -- +# on a large table that falls back to copy, --json refuses without --force. +# See workspace-workflow.md "Load types" for the full clone/copy/view rules. kbagent workspace load --workspace-id W --tables in.c-foo.data kbagent workspace query --workspace-id W --sql " CREATE TABLE \"in.c-foo.data_change_log\" AS diff --git a/plugins/kbagent/skills/kbagent/references/typify-table-workflow.md b/plugins/kbagent/skills/kbagent/references/typify-table-workflow.md index a8505b85..e4bc484b 100644 --- a/plugins/kbagent/skills/kbagent/references/typify-table-workflow.md +++ b/plugins/kbagent/skills/kbagent/references/typify-table-workflow.md @@ -96,7 +96,12 @@ kbagent --json workspace create --project ALIAS --name "typify-" # -> workspace_id, host, database, schema, user, password # SAVE THE PASSWORD -- it cannot be retrieved later. -# 2b. Load the typeless table +# 2b. Load the typeless table. Default (no --load-type) auto-picks clone +# when eligible -- zero-copy, seconds, any size. If it falls back to copy +# (see tables[].clone_ineligible_reason in the response) and the table is +# over 1 GiB, --json mode refuses without --force (no prompt to answer) -- +# add --force once you've confirmed the copy cost is expected. See +# workspace-workflow.md "Load types" for the full clone/copy/view rules. kbagent --json workspace load \ --project ALIAS \ --workspace-id W_ID \ diff --git a/plugins/kbagent/skills/kbagent/references/workspace-workflow.md b/plugins/kbagent/skills/kbagent/references/workspace-workflow.md index c422fb87..b955b697 100644 --- a/plugins/kbagent/skills/kbagent/references/workspace-workflow.md +++ b/plugins/kbagent/skills/kbagent/references/workspace-workflow.md @@ -75,12 +75,23 @@ kbagent --json workspace load \ --tables in.c-bucket.table3 \ --preserve -# Force a zero-copy CLONE and skip the size guard on a very large table +# Force clone explicitly -- fails loudly (HTTP 400) instead of silently +# falling back to copy if the table turns out not to be clone-eligible. +# --force is irrelevant here: the size guard only ever applies to copy. +kbagent --json workspace load \ + --project ALIAS \ + --workspace-id WS_ID \ + --tables in.c-bucket.table1 \ + --load-type clone + +# Force a real COPY of a table you know is huge and not clone-eligible -- +# --force is required (there is no prompt under --json), and a COPY really +# moves the data so it can outrun the 300s default --timeout. kbagent --json workspace load \ --project ALIAS \ --workspace-id WS_ID \ --tables in.c-bucket.huge_table \ - --load-type clone \ + --load-type copy \ --force \ --timeout 600 @@ -93,24 +104,37 @@ kbagent --json workspace query \ ## Load types: clone / copy / view -*(since vNEXT, #687)* `workspace load` supports three load types. **Default (no `--load-type`)**: -kbagent decides per table, mirroring the server's own eligibility rules -- -zero-copy `clone` when the table is on the same backend as the workspace -with a full (unfiltered) load and no external-schema/Analytics-Hub source -bucket, otherwise a plain `copy`. Check `tables[].clone_ineligible_reason` -in the JSON to see why a given table fell back to `copy`. +*(since vNEXT, #687)* **Cheapest first**: skip the load entirely if you can +(see "Don't load at all" below) > `clone` (zero-copy, seconds, any table +size) > `view` (zero-storage, narrower eligibility) > `copy` (real data +movement, size-guarded). `workspace load` supports all three as +`--load-type`. **Default (no `--load-type`)**: kbagent decides per table, +mirroring the server's own eligibility rules -- zero-copy `clone` when the +table is on the same backend as the workspace with a full (unfiltered) load +and no external-schema/Analytics-Hub source bucket, otherwise a plain +`copy`. Check `tables[].clone_ineligible_reason` in the JSON to see why a +given table fell back to `copy`. - **`--load-type clone`** -- zero-copy, metadata-only, finishes in seconds regardless of table size. Fails loudly (server HTTP 400) if any requested table is not clone-eligible -- it does not silently fall back to `copy`. - **`--load-type copy`** -- always a real data copy. A table over 1 GiB - needs interactive confirmation (TTY) or `--force` (JSON / scripted runs) - before the load starts. + needs interactive confirmation or `--force`. The confirmation prompt only + exists in human mode (no `--json`); `--json` (the default per Rule 1 + above) never prompts, so a scripted `--json` load of a table that might + cross 1 GiB needs `--force` up front or it is refused outright. Check + `dataSizeBytes` via `storage table-detail` beforehand if you want to know + which one you'll need without a failed dry run. - **`--load-type view`** -- zero-storage, read-only view, no data movement at all. BigQuery: any same-backend bucket. Snowflake: only an external-schema bucket with the project feature `input-mapping-read-only-storage`. +**Verify what actually happened**: read `tables[].load_type` and +`tables[].data_size_bytes` in the JSON result rather than assume -- with +the auto-default, two requests that look identical can each take a +different path per table. + Loading a large table can take a while even as a copy job; `--timeout` defaults to 300s for `workspace load` (other storage-job commands keep 60s). If it times out, the storage job **keeps running server-side** -- From 735d5bf84b9639799dfa599fc58275b3c9e0827a Mon Sep 17 00:00:00 2001 From: Petr Date: Tue, 25 Aug 2026 15:19:28 +0200 Subject: [PATCH 2/2] docs(plugin): keep SKILL.md description under the 1024-char Claude Desktop limit test_skill_frontmatter caps the frontmatter description at 1024 chars (issue #447); the new workspace-load triggers pushed it to 1088. Keep the two strongest new triggers (zero-copy clone, workspace load type) and drop the descriptive '(backup/restore)' parenthetical. --- plugins/kbagent/skills/kbagent/SKILL.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/plugins/kbagent/skills/kbagent/SKILL.md b/plugins/kbagent/skills/kbagent/SKILL.md index 9ef95191..9a2f91b4 100644 --- a/plugins/kbagent/skills/kbagent/SKILL.md +++ b/plugins/kbagent/skills/kbagent/SKILL.md @@ -5,7 +5,7 @@ description: > Covers: exploring and searching configurations, job history, data lineage, dev branches, workspace SQL debugging, GitOps config sync (pull/push/diff/clone), bucket sharing and linking, encrypting secrets, - Storage tables, files, and snapshots (backup/restore), data apps + Storage tables, files, and snapshots, data apps (deploy/logs/secrets), flows and schedules, members and invitations, feature flags, OTLP data streams, scoped Storage tokens, the semantic layer (models, metrics), the Developer Portal, browser login. @@ -16,8 +16,7 @@ description: > feature flag, flow schedule, invite member, SQL transformation edit, sync action, keboola docs, table snapshot, auth, login, sign in, PAYG credits, flow notifications, alert recipients, config trash, - restore config, undelete config, zero-copy clone, workspace load type, - table too big to load, load table into workspace cheaply. + restore config, undelete config, zero-copy clone, workspace load type. --- # kbagent -- Keboola Agent CLI