Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,11 @@ kbagent data-app start --project NAME --app-id ID [--wait] [--timeout SECONDS]
kbagent data-app stop --project NAME --app-id ID [--wait] [--timeout SECONDS]
kbagent data-app delete --project NAME --app-id ID [--yes]
kbagent data-app password --project NAME --app-id ID
kbagent data-app secrets-set --project ALIAS --app-id ID --secret '#KEY=VALUE' [--secret ...] [--secrets-file PATH] [--branch ID] [--allow-plaintext-on-encrypt-failure] [--dry-run] [--no-hint-next]
kbagent data-app secrets-list --project ALIAS --app-id ID [--branch ID] [--show-fingerprint]
kbagent data-app secrets-get --project ALIAS --app-id ID --key '#KEY' [--branch ID]
kbagent data-app secrets-remove --project ALIAS --app-id ID --key '#KEY' [--key ...] [--branch ID] [--yes] [--dry-run]
kbagent data-app validate-repo --git-repo URL [--git-branch BRANCH] [--git-public/--no-git-public] [--git-pat-env VAR | --git-pat-file PATH] [--type python-js] [--strict]

kbagent component list [--project NAME] [--type TYPE] [--query QUERY]
kbagent component detail --component-id ID [--project NAME]
Expand Down
47 changes: 46 additions & 1 deletion plugins/kbagent/agents/keboola-expert.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ a critical failure.
`data-app password` needs 0.28.0+ with `--allow-env-manage-token`
(the env var is default-deny on 0.28.0+),
`project invite` / `project member-*` / `project invitation-*`
need 0.26.1+, `storage retype` is a future composite), you
need 0.26.1+,
`data-app secrets-* / validate-repo` need 0.28.0+,
`storage retype` is a future composite), you
MUST refuse the task and return a handoff message to the parent:
`"Cannot proceed safely on kbagent <version>. Missing: <commands>.
Ask user to run kbagent update, then re-invoke me."` Do not attempt
Expand Down Expand Up @@ -119,6 +121,11 @@ a critical failure.
| Cancel a pending invitation | `kbagent project invitation-cancel --project P --email E --yes` (0.26.1+) | `--invitation-id ID` if email lookup is ambiguous | DELETE via raw HTTP without going through the service layer |
| Remove an active member | `kbagent project member-remove --project P --email E --yes` (0.26.1+, **destructive**) | `--hint client` for a script that removes by user_id directly | calling `member-remove` without `--yes` in non-interactive contexts (it will prompt and hang) |
| Change a member's role | `kbagent project member-set-role --project P --email E --role admin\|guest\|readOnly\|share` (0.26.1+) | -- | `PUT /manage/projects/{id}/users/{userId}` -- the API rejects PUT with 404, the kbagent client correctly uses **PATCH** |
| Set / rotate app-runtime secrets | `kbagent data-app secrets-set --project P --app-id N --secret '#KEY=VAL'` (0.28.0+) then `data-app deploy --wait` -- per-project KMS encryption, fail-closed, never auto-deploys | `kbagent encrypt values --component-id keboola.data-apps` + `tool call update_config` -- ONLY if you need to write secrets to a different shape than `parameters.dataApp.secrets` | raw `POST` to encryption + Storage without read-modify-write -- you will clobber sibling keys nested under `parameters.dataApp.secrets` (Storage `merge=True` is shallow at the top level only) |
| Inspect what secrets are set on a data app | `kbagent data-app secrets-list --project P --app-id N` (0.28.0+) -- metadata only, never decrypts | `tool call get_configs --component_id keboola.data-apps` then read `parameters.dataApp.secrets` keys (raw dict, no env-var derivation, may leak ciphertext into output) | trying to decrypt -- the Encryption API has no decrypt endpoint, the CLI cannot decrypt under any branch |
| Confirm one secret is present | `kbagent data-app secrets-get --project P --app-id N --key '#KEY'` (0.28.0+) -- returns metadata only | -- | trying to extract the plaintext value (impossible by design; not a CLI gap) |
| Remove a secret from a data app | `kbagent data-app secrets-remove --project P --app-id N --key '#KEY' --yes` (0.28.0+) -- idempotent; missing keys exit 0 with `removed: 0` | `tool call update_config` with the secrets sub-dict deleted -- ONLY for batch removes that need a custom change description | `kbagent config update --set 'parameters.dataApp.secrets={}'` -- replaces the whole sub-dict, dropping every secret instead of just the named ones |
| Pre-flight a data-app repo before create | `kbagent data-app validate-repo --git-repo URL --type python-js [--git-pat-env VAR]` (0.28.0+) -- BLOCKING / WARN / OK with help-doc citations; ≤5 GitHub API calls regardless of repo size | git-clone the repo locally and inspect by hand | `data-app create --dry-run` (only shows the request bodies; does not validate repo structure) |

If the table does not cover the user's task, **ask clarifying
questions** instead of guessing. Returning a targeted question is a
Expand Down Expand Up @@ -290,6 +297,44 @@ success, not a failure.
the target project's Encryption API and refuses to write plaintext if
the round-trip does not return a `KBC::Project*` ciphertext.

- **`data-app create --auth public` writes the canonical `noneProxyAuthorization`
shape** (0.28.0+, fixes a v0.27.0 silent-503 bug): v0.27.0 wrote NO
`authorization` block when `--auth public` -- the Keboola app-proxy
refused to route (HTTP 503) and the UI's Authentication Type selector
showed blank. v0.28.0 writes
`{auth_providers: [], auth_rules: [{type: pathPrefix, value: /, auth_required: false}]}`
per the kbc-ui's `noneProxyAuthorization` constant. If a user reports a
v0.27.0 public app returning 503, the fix is to recreate on 0.28.0+
(the URL is bound to the deployment record so it retires either way),
OR to patch the existing config in-place via
`kbagent config update --component-id keboola.data-apps --config-id ID
--set 'authorization=...'` with the canonical shape. `--auth password`
behaviour is unchanged. Other auth providers (OIDC / GitHub / GitLab /
JumpCloud / Auth0) are not yet exposed by the CLI; tracked as a
follow-up issue.

- **`data-app secrets-* metadata-only`** (0.28.0+): `secrets-get` NEVER
echoes the decrypted plaintext under any branch -- the Encryption API
is one-way and the CLI does not attempt to decrypt. NOT_FOUND on an
absent key never enumerates sibling keys (avoids leaking neighbour
presence). `secrets-remove` is idempotent: removing a non-existent key
returns exit 0 with `removed: 0` and does NOT bump the Storage
version. Setting a key whose derived env-var name collides with the
runtime-injected set (`KBC_TOKEN`, `KBC_URL` for sure; more TODO) is
silently shadowed by the platform; the CLI emits a stderr WARN and
surfaces `shadowed_by_runtime` in JSON envelope -- the WRITE still
happens. Read-modify-write is at the SERVICE layer (Storage `merge=True`
is shallow at the top level only and would clobber siblings nested in
`parameters.dataApp.secrets`).

- **`data-app validate-repo` is GitHub-only**, `--type python-js` only
(0.28.0+): pre-flight Golden-Rule check via the GitHub Trees+Contents
API. Total <=5 calls regardless of repo size. Use BEFORE
`data-app create` so the operator does not burn a deploy cycle on a
misconfigured repo. WARNs are advisory unless `--strict` is set;
BLOCKINGs always fail. Tracked follow-up: streamlit / pure-Python /
R / Node-only types, GitLab/Bitbucket hosts.

- **`storage bucket-detail` is dialect-aware** (0.25.3+): the response
shape depends on the bucket's backend. Snowflake buckets carry
`snowflake_database` / `snowflake_schema` and per-table
Expand Down
5 changes: 5 additions & 0 deletions plugins/kbagent/skills/kbagent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,11 @@ When working inside a git repository or project directory, run `kbagent init` (o
| Stop a running data app (preserves the URL and Storage config) | `kbagent data-app stop --project PROJECT --app-id APP-ID` |
| Delete the deployment AND the Storage config (cascade, irreversible) | `kbagent data-app delete --project PROJECT --app-id APP-ID` |
| Retrieve the simpleAuth password for a password-gated data app | `kbagent data-app password --project PROJECT --app-id APP-ID` |
| Encrypt and write app-runtime secrets to the linked Storage config | `kbagent data-app secrets-set --project PROJECT --app-id APP-ID` |
| List the keys in parameters.dataApp.secrets, with derived runtime env-var names | `kbagent data-app secrets-list --project PROJECT --app-id APP-ID` |
| Show metadata for ONE secret key. | `kbagent data-app secrets-get --project PROJECT --app-id APP-ID --key KEY` |
| Remove one or more app-runtime secrets. | `kbagent data-app secrets-remove --project PROJECT --app-id APP-ID --key KEY` |
| Pre-flight check that a git repo follows the Keboola data-app Golden Rule | `kbagent data-app validate-repo --git-repo GIT-REPO` |
| List jobs from connected projects | `kbagent job list` |
| Show detailed information about a specific job | `kbagent job detail --project PROJECT --job-id JOB-ID` |
| Run a job for a component configuration | `kbagent job run --project PROJECT --component-id COMPONENT-ID --config-id CONFIG-ID` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,11 @@ Lifecycle for `keboola.data-apps`. Combines Storage API (config body, git block,
- `data-app stop --project NAME --app-id ID [--wait] [--timeout SECONDS]` -- stop a running app (URL and Storage config preserved).
- `data-app delete --project NAME --app-id ID [--yes]` -- destructive, cascades to Storage config; URL retired permanently.
- `data-app password --project NAME --app-id ID` -- read the simpleAuth password. Manage token via interactive prompt by default, or `--allow-env-manage-token` + `KBC_MANAGE_API_TOKEN` for CI on 0.28.0+. Auto-generated, not rotatable -- delete + recreate to mint a new one.
- `data-app secrets-set --project ALIAS --app-id ID --secret '#KEY=VALUE' [--secret ...] [--secrets-file PATH] [--branch ID] [--allow-plaintext-on-encrypt-failure] [--dry-run] [--no-hint-next]` -- encrypt and write `#`-prefixed secrets to `parameters.dataApp.secrets`. Per-project KMS encryption, fail-closed. Read-modify-write at the service layer (NOT Storage `merge=True` -- shallow). Runtime exposes each key as an env var with `#` stripped, `-` -> `_`, uppercased. Adding bumps the Storage version; the running container keeps the OLD config until the next `data-app deploy`.
- `data-app secrets-list --project ALIAS --app-id ID [--branch ID] [--show-fingerprint]` -- list secret keys + derived runtime env-var names. Never echoes encrypted ciphertext in full. `--show-fingerprint` opt-in for a short ciphertext fingerprint.
- `data-app secrets-get --project ALIAS --app-id ID --key '#KEY' [--branch ID]` -- show metadata for ONE secret. NEVER echoes the decrypted value (Encryption API is one-way). NOT_FOUND on absent key; never enumerates siblings.
- `data-app secrets-remove --project ALIAS --app-id ID --key '#KEY' [--key ...] [--branch ID] [--yes] [--dry-run]` -- destructive (can break a running app at next deploy). Idempotent: missing keys exit 0 with `removed: 0`.
- `data-app validate-repo --git-repo URL [--git-branch BRANCH] [--git-public/--no-git-public] [--git-pat-env VAR | --git-pat-file PATH] [--type python-js] [--strict]` -- pre-flight Golden-Rule check for a data-app git repo (https://help.keboola.com/data-apps/python-js/). GitHub-only; ≤5 API calls (1 tree + ≤4 contents) regardless of repo size. `--type` restricted to `python-js` in 0.28.0; streamlit / pure-Python / R / Node-only follow-up. `--strict` treats WARNs as failures.

## MCP Tools
- `tool list [--project NAME] [--branch ID]` -- list available MCP tools (multi_project annotation)
Expand Down
79 changes: 75 additions & 4 deletions plugins/kbagent/skills/kbagent/references/data-app-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,68 @@ kbagent data-app deploy --project prod --app-id 12345678 \
(rollback). Subsequent deploys without the flag will jump back to the
latest.

### Pre-flight repo validation (since v0.28.0)

```bash
kbagent data-app validate-repo \
--git-repo https://github.com/myorg/dashboard \
--git-branch main \
--git-pat-env GITHUB_PAT_DATAAPP \
--type python-js
```

Walks the repo via the GitHub Contents + Trees API and emits
BLOCKING / WARN / OK per check (Golden-Rule structure, no `pip install`
in `setup.sh`, `requires-python` consistency, nginx/app port match,
etc.). Each check carries a citation back to the help-doc anchor
(<https://help.keboola.com/data-apps/python-js/>). Run before
`data-app create` so you don't burn a deploy cycle on a misconfigured
repo. Public repos: drop `--git-pat-env` and use `--git-public`. Total
GitHub call budget per run is ≤5 (1 tree + ≤4 contents) regardless of repo size, so the
60/hour unauth limit rarely fires; pass a PAT for CI loops.

### Manage app-runtime secrets (since v0.28.0)

```bash
# Set two secrets at once. Plaintext values; the CLI encrypts under
# THIS project's KMS via the Encryption API before writing to Storage.
kbagent --json data-app secrets-set \
--project prod --app-id 12345678 \
--secret '#ANTHROPIC_API_KEY=sk-ant-...' \
--secret '#my-database-url=postgres://...'

# Then redeploy so the running container picks up the new env. The
# JSON envelope from secrets-set carries a `next_step` field with the
# exact command; suppress it with --no-hint-next for scripted callers.
kbagent data-app deploy --project prod --app-id 12345678 --wait

# Inspect what's set without echoing the encrypted ciphertext:
kbagent data-app secrets-list --project prod --app-id 12345678
# -> #ANTHROPIC_API_KEY -> env ANTHROPIC_API_KEY
# -> #my-database-url -> env MY_DATABASE_URL

# Confirm presence of one key (NEVER decrypts):
kbagent data-app secrets-get --project prod --app-id 12345678 --key '#ANTHROPIC_API_KEY'

# Remove (idempotent -- absent keys exit 0 with removed=0):
kbagent data-app secrets-remove --project prod --app-id 12345678 \
--key '#my-database-url' --yes
```

The runtime exposes each secret as an env var with `#` stripped, `-`
replaced with `_`, and uppercased
(<https://help.keboola.com/data-apps/python-js/>). `secrets-set` does
read-modify-write at the service layer (Storage `merge=True` is
shallow at the top level only and would clobber siblings nested under
`parameters.dataApp.secrets`); every untouched key in the config body
is preserved bit-identical. Encryption is per-project KMS, fail-closed:
if the Encryption API does not return a `KBC::Project*` ciphertext,
the command aborts with `ENCRYPTION_FAILED` and Storage is never
written. Setting a key whose derived env-var name collides with the
runtime-injected set (`KBC_TOKEN`, `KBC_URL` for sure; more TODO
follow-up) emits a stderr WARN -- the platform value silently shadows
yours at runtime.

## Gotchas encoded in the CLI (so you don't have to think about them)

1. **§9 redeploy contract** — `data-app deploy` always sends the
Expand Down Expand Up @@ -158,15 +220,24 @@ latest.
| Wake an auto-suspended app | `data-app start --app-id N` |
| Pause a running app temporarily | `data-app stop --app-id N` |
| Read the simpleAuth password | `data-app password --app-id N` (needs Manage token) |
| Set or rotate app-runtime secrets | `data-app secrets-set --app-id N --secret '#KEY=VAL'` then `data-app deploy --wait` |
| Inspect what secrets are set | `data-app secrets-list --app-id N` (metadata only, never decrypts) |
| Confirm one secret is present | `data-app secrets-get --app-id N --key '#KEY'` (metadata only) |
| Remove a secret | `data-app secrets-remove --app-id N --key '#KEY' --yes` (idempotent) |
| Pre-flight a repo before create | `data-app validate-repo --git-repo URL` (GitHub-only, python-js for now) |
| Tear it all down | `data-app delete --app-id N` (cascades to Storage config) |

## What this command group deliberately does NOT cover

- **Reading the build / runtime log** — the Data Science API does not
expose Terminal Logs as JSON; only the Keboola UI ("Terminal Log" tab)
shows them. If `data-app deploy --wait` exits with
`DATA_APP_BUILD_FAILED`, the next step is to open the UI link surfaced
in the error message.
expose Terminal Logs as JSON; only the Keboola UI ("Terminal Log" tab
at https://help.keboola.com/data-apps/terminal-log-tab/) shows them.
If `data-app deploy --wait` exits with `DATA_APP_BUILD_FAILED`, the
next step is to open the UI link surfaced in the error message.
A `data-app logs` command + auto-log-dump on deploy failure are
tracked as a follow-up: see `padak/keboola_agent_cli`
[issue #240](https://github.com/padak/keboola_agent_cli/issues/240)
(needs platform-side API exposure first).
- **Updating size / auto-suspend / git settings** — those live on the
Storage config body, not the deployment record. Use
`kbagent config update --component-id keboola.data-apps --config-id ID
Expand Down
Loading