Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ Requires a **super-admin** Manage API token (same kind as `org setup`). Same def
- `component sync-action ACTION_NAME --component-id ID --project ALIAS (--config-id ID [--row-id ID] | --config-data JSON|@file|-) [--branch ID] [--timeout N]` (since 0.73.0) -- run a synchronous component action (`testConnection`, `getTables`, ...) on the `sync-actions.{stack}` service. `ACTION_NAME` is freeform (component-defined; discover via `component detail` `synchronous_actions`). `--row-id` shallow-merges the row over the root config at TOP level only (row `parameters`/`storage` replace root wholesale -- NOT deep merge; MCP `run_sync_action` parity). `--config-data` sends explicit `configData` verbatim. Response is action-specific pass-through. Ports the `run_sync_action` MCP tool. **Since 0.89.0 (#620)** the ROOT configuration's `authorization` and `runtime` blocks are forwarded into `configData` too -- root only (a `--row-id` never overrides them), and only when non-empty. `authorization.oauth_api.id` is the OAuth broker reference the sync-actions service resolves and decrypts, so on 0.88.0 and earlier every sync action on an OAuth / Service-Account component (`keboola.ex-linkedin-ads`, ...) failed with an opaque empty-body 400.
- `config examples --component-id ID [--project NAME] [--row]` (since 0.73.0) -- sample root/row configurations from the AI-service component detail. `--json` emits `{component_id, root_examples, row_examples}`; `--row` limits to row examples. Ports the `get_config_examples` MCP tool.
- `component list [--project NAME] [--type TYPE] [--query "text"]` -- list/search components (AI-powered with `--query`)
- `component detail --component-id ID [--project NAME]` -- show component schema, docs URL, examples. **Since vNEXT** a component the AI Service does not index (private/deprecated: `keboola.mcp-server-tool`, `keboola.data-apps`) no longer errors -- it falls back to the project's Storage component catalog. `documentation_source` (`"ai_service"` vs `"storage_catalog"`) is on BOTH paths and tells them apart; the fallback has NO configuration examples (`examples_count`/`row_examples_count` always 0), so read `documentation_source` before treating 0 as "this component ships none". `NOT_FOUND` is still raised when both sources miss. See `gotchas.md`.
- `component detail --component-id ID [--project NAME]` -- show component schema, docs URL, examples. **Since vNEXT** a component the AI Service does not index (private/deprecated: `keboola.mcp-server-tool`, `keboola.data-apps`) no longer errors -- it falls back to the project's Storage component catalog. `documentation_source` (`"ai_service"` vs `"storage_catalog"`) is on BOTH paths and tells them apart; the fallback has NO configuration examples (`examples_count`/`row_examples_count` always 0), so read `documentation_source` before treating 0 as "this component ships none". `NOT_FOUND` is still raised when both sources miss. **Since vNEXT** omitting `--project` really does use the first configured project -- on <= 0.89.x it failed with `CONFIG_ERROR: Project 'None' not found` despite the help text, so pass `--project` explicitly there. See `gotchas.md`.

## Configuration Browsing
- `config list [--project NAME] [--component-type TYPE] [--component-id ID] [--branch ID] [--include-rows]` -- list configs across projects (branch-aware). With `--include-rows` each row extends to include the full `configuration` and `rows` body (noticeably larger payload -- use only when the bodies are needed; the summary default covers name/description/component/last_modified/folder)
Expand All @@ -131,7 +131,7 @@ Requires a **super-admin** Manage API token (same kind as `org setup`). Same def
- `config delete --project NAME --component-id ID --config-id ID [--branch ID] [--dry-run]` -- SOFT-delete a configuration into the Storage trash (restorable). Since 0.89.0 it locates the config first and a config already in the trash is NOT deleted again -- the raw API purges permanently on a second DELETE (the retry-after-timeout trap); kbagent reports `already_in_trash` and exits 0 instead. `--dry-run` reports the located state without writing. Undo with `config restore`. Permission class `destructive`
- `config restore --project NAME --component-id ID --config-id ID [--branch ID]` *(since v0.89.0)* -- restore a trashed configuration (versions, rows and metadata come back). Only works on a config currently in the trash. Permission class `write`
- `config trash-list --project NAME [--component-id ID] [--branch ID]` *(since v0.89.0)* -- list configurations in the trash; each row carries `component_id`, `config_id`, `name`, `version` and `deleted_at`, which is exactly what `config restore` needs. Permission class `read`
- `config new --component-id ID [--project NAME] [--name NAME] [--output-dir DIR] [--push --no-files --description D --configuration JSON|@file|- --configuration-file PATH --no-validate --branch ID --dry-run --allow-plaintext-on-encrypt-failure]` -- **two modes**. **Default (no `--push`)**: scaffold new config from component schema; writes files to `--output-dir` or prints to stdout. **Zero API calls.** **With `--push`** (0.33.0+, requires `--project` + non-empty `--name`): also POSTs to `/v2/storage/components/{cid}/configs` for a one-shot remote create. `#`-prefixed secrets in the pushed body auto-encrypt via the Encryption API first (fail-closed; since 0.54.0, #378; `--allow-plaintext-on-encrypt-failure` overrides). `--no-files` skips the filesystem step entirely (FIIA-style empty-shell pattern). `--configuration` / `--configuration-file` override the POSTed body (default is `{}`, with validation auto-skipped for the default empty shell). `--dry-run` previews the planned POST + validation result without creating. Since 0.89.0, `--push` + `--output-dir` writes the scaffold WITH `_keboola.config_id` into the subtree of the branch the config was created in (registering the branch in the manifest when missing), so the next `sync push` adopts it instead of duplicating (issue #644); with an explicit body the local file mirrors the pushed encrypted configuration. Schema validation runs by default when an explicit body is given (fail-closed: `ConfigError` exit 5 on mismatch) but skips silently if the AI Service has no schema for the component or returns an error; `--no-validate` opts out. Works for ALL component types including `keboola.snowflake-transformation`.
- `config new --component-id ID [--project NAME] [--name NAME] [--output-dir DIR] [--push --no-files --description D --configuration JSON|@file|- --configuration-file PATH --no-validate --branch ID --dry-run --allow-plaintext-on-encrypt-failure]` -- **two modes**. **Default (no `--push`)**: scaffold new config from component schema; writes files to `--output-dir` or prints to stdout. **Zero API calls.** **With `--push`** (0.33.0+, requires `--project` + non-empty `--name`): also POSTs to `/v2/storage/components/{cid}/configs` for a one-shot remote create. `#`-prefixed secrets in the pushed body auto-encrypt via the Encryption API first (fail-closed; since 0.54.0, #378; `--allow-plaintext-on-encrypt-failure` overrides). `--no-files` skips the filesystem step entirely (FIIA-style empty-shell pattern). `--configuration` / `--configuration-file` override the POSTed body (default is `{}`, with validation auto-skipped for the default empty shell). `--dry-run` previews the planned POST + validation result without creating. Since 0.89.0, `--push` + `--output-dir` writes the scaffold WITH `_keboola.config_id` into the subtree of the branch the config was created in (registering the branch in the manifest when missing), so the next `sync push` adopts it instead of duplicating (issue #644); with an explicit body the local file mirrors the pushed encrypted configuration. Schema validation runs by default when an explicit body is given (fail-closed: `ConfigError` exit 5 on mismatch) but skips silently if the AI Service has no schema for the component or returns an error; `--no-validate` opts out. Works for ALL component types including `keboola.snowflake-transformation`. **Since vNEXT** the scaffold mode's optional `--project` really does fall back to the first configured project when omitted -- on <= 0.89.x it failed with `CONFIG_ERROR: Project 'None' not found` (see `gotchas.md`).
- `config clone --project P --component-id ID --config-id ID --name NAME [--target-project P2] [--description D] [--set PATH=VALUE ...] [--secret PATH=VALUE ...] [--branch ID] [--target-branch ID] [--dry-run] [--allow-plaintext-on-encrypt-failure]` (0.84.2+, #587) -- duplicate a configuration **whole**. Reach for this instead of reading `config detail` and rebuilding a body: copying only `configuration["parameters"]` silently drops its siblings (`runtime`, `storage`, `authorization`), and a lost `runtime.parallelism` makes Keboola fall back to `parallelism: 1` -- the reporter's 65-row writer went sequential, 140 min instead of ~60-90, with nothing in any output pointing at it. **Same project** (default): server-side copy via `POST .../configs/{id}/versions/{v}/create`; rows and `KBC::` encrypted values travel with it (verified live). `--set PATH=VALUE` is applied as a follow-up update on the copy, so an override can never be the reason a key went missing. **Cross project** (`--target-project`): reassembled client-side and rows recreated one by one, because encrypted values **cannot** travel -- a Keboola ciphertext is scoped to the project it was encrypted in. Any `KBC::` value makes the clone **fail with exit 5**, listing every path, until re-supplied via `--secret PATH=VALUE` (encrypted in the TARGET project on write). `--dry-run` reports those paths instead of refusing -- run it first to learn what to gather. Storage bucket/table IDs are copied **verbatim, never remapped**; `sync clone` is the command that remaps.
- `config variables-set --project NAME --component-id ID --config-id ID --var KEY=VALUE [--var ...] [--replace] [--variables-id ID] [--values-id ID] [--branch ID] [--dry-run] [--allow-plaintext-on-encrypt-failure] [--yes]` -- attach variable values to a config. Auto-creates a sibling `keboola.variables` config + default row on first use and links it via the parent's `runtime.variables_id` / `variables_values_id`. Defaults to merge; `--replace` drops keys not in `--var`. `#`-prefixed values encrypt via the Encryption API (fail-closed; exit non-zero on `ENCRYPTION_FAILED`). See `variables-workflow.md`
- `config variables-get --project NAME --component-id ID --config-id ID [--branch ID]` -- resolve `variables_id` + `values_id` from the parent config and fetch the current KEY=VALUE map. Returns `{linked: bool, variables_id, values_id, values}`; `linked=false` means the parent has no variables attached
Expand Down
20 changes: 20 additions & 0 deletions plugins/kbagent/skills/kbagent/references/gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -4412,3 +4412,23 @@ looked like an upstream outage worth retrying.
- **Over `serve`, a `KeboolaApiError` with code `NOT_FOUND` now answers HTTP
404, not 502** (all routers, not just components). Branch on
`error.code`, not on the HTTP status alone.

## `component detail` / `config new` without `--project` really uses the first project (since vNEXT)

`component detail --component-id ID` and `config new --component-id ID`
(scaffold-only, no `--push`) have always documented `--project` as optional
("uses first available if not set" / "for AI Service auth; required with
--push"). On every version before vNEXT that promise was broken: omitting the
flag failed with `CONFIG_ERROR: Project 'None' not found ...`, because the
omitted alias was passed into project resolution as a literal `None` element
and took the strict-lookup path instead of the "first available project"
fallback (`config examples` already resolved it correctly).

- **vNEXT+**: omitting `--project` resolves to the first configured project;
`component detail`'s `project_alias` reports the alias actually used (never
`None`). With NO projects configured at all, the failure is an actionable
`CONFIG_ERROR: No projects configured. Use 'kbagent project add' ...`.
- **<= 0.89.x**: pass `--project` explicitly to these two commands -- the help
text's "first available" promise does not work there.
- `component sync-action` is unaffected: its `--project` is genuinely required
(exit 2 without it) on every version.
73 changes: 54 additions & 19 deletions src/keboola_agent_cli/services/component_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import logging
from collections.abc import Callable
from dataclasses import dataclass
from pathlib import Path
from typing import Any

Expand All @@ -27,6 +28,20 @@
AiClientFactory = Callable[[str, str], AiServiceClient]


@dataclass(frozen=True)
class ResolvedProject:
"""A project resolved together with the alias it was resolved under.

``alias`` is the caller's alias when one was given, otherwise the first
configured project's -- the value response payloads report as
``project_alias``. A dataclass rather than a bare 2-tuple per
CONTRIBUTING.md's multi-value-return rule.
"""

alias: str
project: ProjectConfig


def default_ai_client_factory(stack_url: str, token: str) -> AiServiceClient:
"""Create an AiServiceClient with the given stack URL and token.

Expand Down Expand Up @@ -522,7 +537,29 @@ def list_components(
return self._list_via_ai(aliases, component_type, query)
return self._list_via_storage(aliases, component_type)

def get_component_detail(self, alias: str, component_id: str) -> dict[str, Any]:
def _resolve_alias_or_first(self, alias: str | None) -> ResolvedProject:
"""Resolve *alias* to a project, defaulting to the first configured one.

:meth:`BaseService.resolve_projects` falls back to "all projects" only
when the alias LIST itself is empty/None -- a ``[None]`` element goes
down its strict lookup and raises "Project 'None' not found". Commands
that document ``--project`` as optional (``component detail``,
``config examples``, ``config new`` without ``--push``) must therefore
normalise an omitted alias here instead of passing it through.

Raises:
ConfigError: If *alias* is given but unknown, or when no projects
are configured at all.
"""
projects = self.resolve_projects([alias] if alias else None)
if not projects:
raise ConfigError(
"No projects configured. Use 'kbagent project add' to connect a project first."
)
resolved_alias = alias or next(iter(projects))
return ResolvedProject(alias=resolved_alias, project=projects[resolved_alias])

def get_component_detail(self, alias: str | None, component_id: str) -> dict[str, Any]:
"""Fetch detailed component documentation, AI Service first.

The AI Service (``/docs/components/{id}``) indexes the PUBLIC component
Expand All @@ -546,21 +583,23 @@ def get_component_detail(self, alias: str, component_id: str) -> dict[str, Any]:
itself -- only the ambiguous 404 is worth a second lookup.

Args:
alias: Project alias (used to derive stack URL and token).
alias: Project alias (used to derive stack URL and token). When
None, the first available project is used.
component_id: The component identifier (e.g. 'keboola.ex-aws-s3').

Returns:
Dict with component detail including schema summary, example
counts, ``documentation_source``, and full documentation.

Raises:
ConfigError: If the alias is not found.
ConfigError: If the alias is not found or no projects are
configured.
KeboolaApiError: If the AI Service call fails; a NOT_FOUND is
re-raised unchanged only when the Storage catalog does not know
the component either (i.e. the id really is wrong).
"""
projects = self.resolve_projects([alias])
project = projects[alias]
resolved = self._resolve_alias_or_first(alias)
project = resolved.project

ai_client = self._ai_client_factory(project.stack_url, project.token)
not_found: KeboolaApiError | None = None
Expand All @@ -578,7 +617,7 @@ def get_component_detail(self, alias: str, component_id: str) -> dict[str, Any]:
catalog_entry = self._find_catalog_component(project, component_id)
if catalog_entry is None:
raise not_found
return self._catalog_detail_payload(catalog_entry, alias)
return self._catalog_detail_payload(catalog_entry, resolved.alias)

detail = ComponentDetail(**raw)

Expand All @@ -603,7 +642,7 @@ def get_component_detail(self, alias: str, component_id: str) -> dict[str, Any]:
},
"examples_count": len(detail.root_configuration_examples),
"row_examples_count": len(detail.row_configuration_examples),
"project_alias": alias,
"project_alias": resolved.alias,
"documentation_source": DOCUMENTATION_SOURCE_AI_SERVICE,
}

Expand Down Expand Up @@ -683,13 +722,7 @@ def get_config_examples(self, alias: str | None, component_id: str) -> dict[str,
ConfigError: If the alias is not found or no projects are configured.
KeboolaApiError: If the AI Service call fails.
"""
projects = self.resolve_projects([alias] if alias else None)
if not projects:
raise ConfigError(
"No projects configured. Use 'kbagent project add' to connect a project first."
)
resolved_alias = alias or next(iter(projects))
project = projects[resolved_alias]
project = self._resolve_alias_or_first(alias).project

ai_client = self._ai_client_factory(project.stack_url, project.token)
try:
Expand Down Expand Up @@ -812,7 +845,7 @@ def run_sync_action(

def generate_scaffold(
self,
alias: str,
alias: str | None,
component_id: str,
name: str | None = None,
) -> dict[str, Any]:
Expand All @@ -822,7 +855,9 @@ def generate_scaffold(
configuration files based on component type and schema.

Args:
alias: Project alias (used to derive stack URL and token).
alias: Project alias (used to derive stack URL and token). When
None, the first available project is used (`config new`
without --push documents --project as optional).
component_id: The component identifier.
name: Configuration name. If None, defaults to
"{component_name} Configuration".
Expand All @@ -831,11 +866,11 @@ def generate_scaffold(
Dict with scaffold metadata and generated files list.

Raises:
ConfigError: If the alias is not found.
ConfigError: If the alias is not found or no projects are
configured.
KeboolaApiError: If the AI Service call fails.
"""
projects = self.resolve_projects([alias])
project = projects[alias]
project = self._resolve_alias_or_first(alias).project

ai_client = self._ai_client_factory(project.stack_url, project.token)
try:
Expand Down
Loading