Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion plugins/kbagent/agents/keboola-expert.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: keboola-expert
description: Keboola Connection operations specialist. MUST BE USED proactively for any task touching Keboola projects -- config browsing/updates, jobs, flows, schedules, storage, migrations, dev branches, debugging. Enforces fresh-fetch discipline, --dry-run on writes, CLI over REST, and refuses tasks it cannot safely complete with the installed kbagent version. Delegates write operations through two-step (dry-run -> confirm -> apply) flow without exception.
description: Keboola Connection ops specialist. Enforces fresh-fetch, dry-run, CLI-over-REST, version gate and confirmed apply.
tools: Bash, Read, Edit, Write, Grep, Glob, TodoWrite, WebFetch
model: sonnet
color: blue
Expand Down Expand Up @@ -111,6 +111,7 @@ a critical failure.
`data-app logs` = 0.43.8+,
`kbagent agent <verb>` (CLI parity /agents REST) = 0.44.0+,
`semantic-layer search-context|get-context`, `storage create-table --if-not-exists`, `sync push|pull|diff --branch`, `sync push --no-name-drift-warnings`, fresh-CREATE writeback + KBC.* = 0.47.0+,
Snowflake `workspace create` `private_key` = 0.47.1+,
`storage retype` is a future composite), you
MUST refuse the task and return a handoff message to the parent:
`"Cannot proceed safely on kbagent <version>. Missing: <commands>.
Expand Down Expand Up @@ -219,6 +220,11 @@ success, not a failure.
lower-level schema. `config new --push` does NOT inherit the MCP
refusal because it calls Storage API directly.

- **Snowflake workspace credentials** (0.47.1+): headless
`kbagent workspace create` returns `private_key` for Snowflake; `password`
is empty/unusable. Use the one-time PKCS8 PEM for key-pair auth. BigQuery
keeps the prior shape.

- **`script[]` string-vs-array runtime crash** (0.28.0+ auto-fix; #245):
the Storage API silently accepts `parameters.blocks[].codes[].script`
as a string, but the runtime validator rejects it (`Expected array,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ All seven commands authenticate via `KBC_MANAGE_API_TOKEN` (Manage API), not the
- `branch metadata-delete --project NAME --metadata-id ID [--branch ID|default]` -- delete a metadata entry by its numeric ID (from `metadata-list`)

## Workspaces (SQL Debugging)
- `workspace create --project ALIAS [--name NAME] [--ui] [--read-only]` -- create workspace (headless ~1s, `--ui` ~15s)
- `workspace create --project ALIAS [--name NAME] [--ui] [--read-only]` -- create workspace (headless ~1s, `--ui` ~15s). Since v0.47.1: Snowflake headless workspaces return a `private_key` PEM field; `password` is empty. BigQuery workspaces keep the default password credential shape.
- `workspace list [--project NAME ...] [--orphaned] [--branch ID] [--qs-compatible]` -- list workspaces. `--project` repeatable; `--orphaned` filters to workspaces whose backing `keboola.sandboxes` config is missing. **Since v0.42.0 (#304)**: each entry carries `login_type`, `read_only`, `qs_compatible`, `database`, `warehouse`. New `Login Type` / `RO` / `QS` columns in human mode. `--qs-compatible` pre-filters to RO + whitelisted-loginType workspaces (the canonical data-app shape). `--branch` requires exactly one `--project`; without `--branch`, the command behaves like `storage buckets` and uses production with an `Info: Using production branch for read (active dev branch X ignored; pass --branch X to override)` banner when an alias is pinned to a dev branch
- `workspace detail --project ALIAS --workspace-id ID [--branch ID]` -- show connection details. **Since v0.42.0 (#304)**: response carries `login_type`, `read_only`, `qs_compatible`; human mode adds `Login type:` / `Read-only:` / `Query Service compatible:` rows. `--branch` opt-in mirrors `workspace list`
- `workspace delete --project ALIAS --workspace-id ID` -- delete workspace
Expand Down
23 changes: 23 additions & 0 deletions plugins/kbagent/skills/kbagent/references/gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ plus a derived `qs_compatible: bool`.

- `snowflake-service-keypair` -- confirmed PASS
- `snowflake-person-sso` -- confirmed PASS
- `snowflake-person-keypair` -- confirmed PASS (since v0.47.1)
- `snowflake-legacy-service` -- explicitly OFF the list (works on
`connection.keboola.com` but FAILED on GCP us-east4 stack in the
original #304 incident -- keep it off until cross-stack confirmation)
Expand All @@ -159,6 +160,28 @@ confirmed-good whitelist". For an unknown loginType, `workspace list`
renders it as `?` (yellow) in the QS column so callers know the policy
is uncertain rather than confirmed-bad.

## Snowflake `workspace create` returns `private_key`, not password (since v0.47.1)

Headless `workspace create` on Snowflake requests
`loginType: snowflake-person-keypair`, generates an RSA key pair locally,
passes the public key to the Storage API, and returns the private key once in
the creation envelope:

```jsonc
{
"backend": "snowflake",
"user": "KEBOOLA_WORKSPACE_42",
"password": "",
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"
}
```

For successful Snowflake creates, `private_key` is the credential to save and
use for key-pair authentication; `password` remains in the envelope for
backward compatibility but should be treated as empty/unusable. BigQuery
workspaces keep the previous password-based/default backend shape and do not
return `private_key`.

**Filter (data-app pre-selection):**

```bash
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ dependencies = [
"prompt-toolkit>=3.0",
"kai-client>=0.11.0",
"croniter>=2.0",
"cryptography>=46",
]

[project.optional-dependencies]
Expand Down
1 change: 1 addition & 0 deletions src/keboola_agent_cli/changelog.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
CHANGELOG: dict[str, list[str]] = {
"0.47.1": [
'Fix (`storage create-table --if-not-exists`, keboola/cli#349): the `action: "skipped"` envelope now reports the EXISTING table\'s actual schema instead of re-echoing the caller\'s request. Pre-0.47.1, `columns` / `primary_key` / `name` on a skip mirrored the args the caller passed in, so a caller probing the skipped envelope to discover the real shape of a pre-existing table got the wrong values whenever the existing table differed from the request. The `get_table_detail(target_id)` lookup that already runs to confirm the table exists is now also the source of the returned schema. The caller\'s requested values are preserved under two new fields, `requested_columns` and `requested_primary_key`, and a new `schema_drift: bool` flags when the existing table diverges from the request (set comparison on columns and primary key). Human-mode output prints the actual schema on a skip and emits a `Warning:` line when `schema_drift` is true. `action: "created"` envelope is unchanged. No new flag, no signature change. Tests: `tests/test_storage_write.py` (skipped returns actual schema, drift flag set on divergence, no drift on match, human-mode warning render); `tests/test_e2e.py::TestE2E_0_47_0_NewSurfaces` extended to assert the skipped envelope reports actual columns + `requested_*` mirror.',
'Fix (`workspace create`, keboola/cli#351): new Snowflake sandbox workspaces now request `loginType: "snowflake-person-keypair"` and generate a local RSA key pair for the Storage API `publicKey` field, so the created workspace uses the Query-Service-compatible login type instead of the backend default. The one-time creation envelope now includes `private_key` for Snowflake workspaces and keeps `password` for compatibility, usually empty on key-pair workspaces; human output prints the private key and warns that it cannot be retrieved later. BigQuery workspaces still omit `loginType` and `publicKey`. Tests cover the Storage client payload, service-layer Snowflake/BigQuery branching, CLI JSON/human output, and Snowflake E2E `private_key` presence.',
],
"0.47.0": [
"Fix (sync push, fresh-CREATE): pre-existing placeholder manifest entries -- the FIIA / scaffold emit pattern, where a downstream tool seeds `.keboola/manifest.json` with placeholder ids and (optionally) `KBC.configuration.*` metadata before the first push -- are now updated **in place** by the create path instead of unconditionally appended. Pre-0.47.0 every create did `manifest.configurations.append(ManifestConfiguration(...))` (and `parent.rows.append(...)` for rows), so N placeholders -> 2N manifest entries after one push, every placeholder still looked `added` on re-push (spurious duplicates on remote), and any `metadata.KBC.configuration.folderName` declared in the placeholder was silently dropped on the floor. Two new private helpers do the work: `SyncService._writeback_create_config_in_manifest(...)` finds the placeholder by `(branch_id, component_id, path)` -- branch is part of the key so a multi-branch manifest with the same logical path under two branches updates the right entry -- and refreshes its id + pull_hash / pull_config_hash while preserving every non-bookkeeping metadata key; `SyncService._writeback_create_row_in_manifest(...)` does the same for rows under their parent. Idempotency on re-push falls out for free: the now-real config id flows through the existing diff engine and the second push reports `status: no_changes, created: 0`. Tests: `tests/test_sync_service.py::TestFreshCreateWriteback` (7 cases incl. an end-to-end placeholder + KBC-metadata round-trip). Manifest contract change for downstream parsers: a single CREATE now produces a single manifest entry (not placeholder + new). Downstream tooling that has been working around the duplication by post-processing must drop that workaround. Live-validated against project 1143 / dev branch 388071: placeholder with `KBC.configuration.folderName: 'Area B E2E Folder'` -> `created=1, errors=0`, manifest length 1, folderName visible via `config metadata-list`, re-push -> `no_changes`.",
Expand Down
12 changes: 11 additions & 1 deletion src/keboola_agent_cli/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -2531,6 +2531,8 @@ def create_config_workspace(
component_id: str,
config_id: str,
backend: str = "snowflake",
login_type: str | None = None,
public_key: str | None = None,
) -> dict[str, Any]:
"""Create a workspace tied to a specific configuration.

Expand All @@ -2539,16 +2541,24 @@ def create_config_workspace(
component_id: Component ID (e.g. keboola.snowflake-transformation).
config_id: Configuration ID.
backend: Workspace backend.
login_type: Optional Storage API loginType. Omitted when None.
public_key: Optional public key for key-pair workspaces. Omitted when None.

Returns:
Workspace dict including connection credentials.
"""
safe_component = quote(component_id, safe="")
safe_config = quote(config_id, safe="")
payload: dict[str, Any] = {"backend": backend}
if login_type is not None:
payload["loginType"] = login_type
if public_key is not None:
payload["publicKey"] = public_key

response = self._request(
"POST",
f"/v2/storage/branch/{branch_id}/components/{safe_component}/configs/{safe_config}/workspaces",
json={"backend": backend},
json=payload,
)
return response.json()

Expand Down
1 change: 1 addition & 0 deletions src/keboola_agent_cli/commands/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -610,6 +610,7 @@

kbagent workspace create --project ALIAS [--name NAME] [--backend TYPE] [--ui] [--read-only/--no-read-only]
Create workspace. Backend auto-detected from project (or override with --backend). Default: headless (~1s). --ui: visible in KBC UI (~15s).
Since 0.47.1, Snowflake headless creates return private_key and an empty password field; use key-pair auth.

kbagent workspace list [--project NAME] [--orphaned] [--branch ID] [--qs-compatible]
List workspaces. Read command: ignores active dev branch (production endpoint) with an Info banner;
Expand Down
20 changes: 16 additions & 4 deletions src/keboola_agent_cli/commands/workspace.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,15 @@ def workspace_create(
c.print(f"[bold]Host:[/bold] {d['host']}"),
c.print(f"[bold]Schema:[/bold] {d['schema']}"),
c.print(f"[bold]User:[/bold] {d['user']}"),
c.print(f"[bold yellow]Password:[/bold yellow] {d['password']}"),
c.print(
"\n[bold yellow]Warning:[/bold yellow] Save the password now -- it cannot be retrieved later!"
f"[bold yellow]Private key:[/bold yellow]\n{d['private_key']}"
if d.get("private_key")
else f"[bold yellow]Password:[/bold yellow] {d['password']}"
),
c.print(
"\n[bold yellow]Warning:[/bold yellow] Save the private key now -- it cannot be retrieved later!"
if d.get("private_key")
else "\n[bold yellow]Warning:[/bold yellow] Save the password now -- it cannot be retrieved later!"
),
),
)
Expand Down Expand Up @@ -645,10 +651,16 @@ def workspace_from_transformation(
c.print(f"[bold]Host:[/bold] {d['host']}"),
c.print(f"[bold]Schema:[/bold] {d['schema']}"),
c.print(f"[bold]User:[/bold] {d['user']}"),
c.print(f"[bold yellow]Password:[/bold yellow] {d['password']}"),
c.print(
f"[bold yellow]Private key:[/bold yellow]\n{d['private_key']}"
if d.get("private_key")
else f"[bold yellow]Password:[/bold yellow] {d['password']}"
),
c.print(f"[bold]Tables loaded:[/bold] {', '.join(d.get('tables_loaded', []))}"),
c.print(
"\n[bold yellow]Warning:[/bold yellow] Save the password now -- it cannot be retrieved later!"
"\n[bold yellow]Warning:[/bold yellow] Save the private key now -- it cannot be retrieved later!"
if d.get("private_key")
else "\n[bold yellow]Warning:[/bold yellow] Save the password now -- it cannot be retrieved later!"
),
),
)
Expand Down
3 changes: 3 additions & 0 deletions src/keboola_agent_cli/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -303,14 +303,17 @@
# Verified 2026-05-18 against project 901 on connection.keboola.com:
# snowflake-service-keypair: PASS
# snowflake-person-sso: PASS
# snowflake-person-keypair: PASS (required for new Snowflake sandboxes)
# snowflake-legacy-service: PASS here, FAIL on GCP us-east4 (issue #304)
# default (legacy 2016 ws): FAIL ('JWT token is invalid')
#
# Extend ONLY after empirical confirmation across at least one non-AWS stack.
SNOWFLAKE_WORKSPACE_LOGIN_TYPE: str = "snowflake-person-keypair"
QUERY_SERVICE_COMPATIBLE_LOGIN_TYPES: frozenset[str] = frozenset(
{
"snowflake-service-keypair",
"snowflake-person-sso",
SNOWFLAKE_WORKSPACE_LOGIN_TYPE,
}
)

Expand Down
7 changes: 6 additions & 1 deletion src/keboola_agent_cli/hints/definitions/workspace.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,11 @@
comment="Create workspace (headless mode)",
client=ClientCall(
method="create_config_workspace",
args={"backend": "{backend}"},
args={
"backend": "{backend}",
"login_type": "login_type",
"public_key": "public_key_pem",
},
result_var="workspace",
result_hint="dict",
),
Expand All @@ -33,6 +37,7 @@
],
notes=[
"Service layer handles sandbox config creation + workspace provisioning.",
"For Snowflake direct client calls, generate an RSA key pair, pass login_type='snowflake-person-keypair' and public_key=public_key_pem, then save the private key returned by your generator. For BigQuery, pass None for both.",
"With --ui flag, creates via job run (slower, ~15s) for UI visibility.",
],
)
Expand Down
Loading
Loading