Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,15 @@ kbagent kai ask --message "question" [--project NAME]
kbagent kai chat --message "msg" [--chat-id ID] [--project NAME]
kbagent kai history [--project NAME] [--limit N]

kbagent flow list [--project NAME] [--branch ID]
kbagent flow detail --project NAME --flow-id ID [--component-id keboola.orchestrator|keboola.flow] [--branch ID]
kbagent flow schema
kbagent flow new --project NAME --name NAME [--component-id keboola.orchestrator|keboola.flow] [--description D] [--file @path.yaml|-|JSON] [--branch ID]
kbagent flow update --project NAME --flow-id ID [--component-id ID] [--name N] [--description D] [--file @path.yaml|-|JSON] [--branch ID]
kbagent flow delete --project NAME --flow-id ID [--component-id ID] [--branch ID] [--yes]
kbagent flow schedule --project NAME --flow-id ID --cron "0 6 * * *" [--component-id ID] [--timezone TZ] [--disabled] [--branch ID]
kbagent flow schedule-remove --project NAME --flow-id ID [--component-id ID] [--branch ID] [--yes]

kbagent context
kbagent init [--from-global]
kbagent doctor [--fix]
Expand Down
2 changes: 1 addition & 1 deletion plugins/kbagent/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "kbagent",
"version": "0.21.1",
"version": "0.22.0",
"description": "AI-friendly interface to Keboola Connection projects — explore configs, jobs, lineage, call MCP tools, manage dev branches, and debug SQL in workspaces",
"author": {
"name": "Keboola",
Expand Down
8 changes: 8 additions & 0 deletions plugins/kbagent/skills/kbagent/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,14 @@ When working inside a git repository or project directory, run `kbagent init` (o
| Ask Kai a one-shot question and get the full response | `kbagent kai ask --message MESSAGE` |
| Send a message to Kai in a chat session | `kbagent kai chat --message MESSAGE` |
| List recent Kai chat sessions | `kbagent kai history` |
| List all flows (keboola.orchestrator + keboola.flow) across projects | `kbagent flow list` |
| Show detailed flow information including phases and tasks | `kbagent flow detail --project PROJECT --flow-id FLOW-ID` |
| Print the YAML format expected by 'flow new' and 'flow update' | `kbagent flow schema` |
| Create a new flow configuration | `kbagent flow new --project PROJECT --name NAME` |
| Update a flow's name, description, or phases/tasks | `kbagent flow update --project PROJECT --flow-id FLOW-ID` |
| Delete a flow configuration | `kbagent flow delete --project PROJECT --flow-id FLOW-ID` |
| Bind a cron schedule to a flow (upsert: creates or updates) | `kbagent flow schedule --project PROJECT --flow-id FLOW-ID --cron CRON` |
| Remove all schedules bound to a flow (deletes keboola.scheduler configs) | `kbagent flow schedule-remove --project PROJECT --flow-id FLOW-ID` |
| List development branches from connected projects | `kbagent branch list` |
| Create a new development branch and auto-activate it | `kbagent branch create --project PROJECT --name NAME` |
| Set an existing development branch as active | `kbagent branch use --project PROJECT --branch BRANCH` |
Expand Down
10 changes: 10 additions & 0 deletions plugins/kbagent/skills/kbagent/references/commands-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,16 @@ All commands support `--json` for structured output. Multi-project flags (`--pro
- `kai chat --message "msg" [--chat-id ID] [--project NAME]` -- send message in a chat session, returns chat_id for continuation
- `kai history [--project NAME] [--limit N]` -- list recent Kai chat sessions (default limit: 10)

## Flows (Orchestrator)
- `flow list [--project NAME] [--branch ID]` -- list all flows (keboola.orchestrator + keboola.flow) across one or all projects
- `flow detail --project NAME --flow-id ID [--component-id keboola.orchestrator|keboola.flow] [--branch ID]` -- full phase/task breakdown; groups tasks by phase, lists orphan tasks
- `flow schema` -- print YAML template for flow configuration (phases + tasks); use with `--file @-` or save to a file
- `flow new --project NAME --name NAME [--component-id keboola.orchestrator|keboola.flow] [--description D] [--file @path.yaml|-|JSON] [--branch ID]` -- create a flow; DAG validated before API call; default component: keboola.flow
- `flow update --project NAME --flow-id ID [--component-id ID] [--name N] [--description D] [--file @path.yaml|-|JSON] [--branch ID]` -- update name, description, or phases/tasks; requires at least one of --name/--description/--file
- `flow delete --project NAME --flow-id ID [--component-id ID] [--branch ID] [--yes]` -- delete a flow config (confirmation guard)
- `flow schedule --project NAME --flow-id ID --cron "0 6 * * *" [--component-id ID] [--timezone TZ] [--disabled] [--branch ID]` -- attach a cron schedule (stored as keboola.scheduler config); replaces any existing schedule
- `flow schedule-remove --project NAME --flow-id ID [--component-id ID] [--branch ID] [--yes]` -- remove all cron schedules attached to a flow; idempotent

## Sync (GitOps)
- `sync init --project ALIAS [--directory DIR] [--git-branching]` -- initialize sync working directory
- `sync pull --project ALIAS [--all-projects] [--force] [--dry-run] [--with-samples] [--no-storage] [--no-jobs] [--job-limit N]` -- download configs to local files. For large projects (>100 configs), automatically fetches jobs per-config when the grouped API limit is insufficient
Expand Down
110 changes: 110 additions & 0 deletions plugins/kbagent/skills/kbagent/references/flow-workflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# Flow Workflow

Flows orchestrate Keboola transformations and extractors in a directed acyclic graph (DAG) of phases and tasks. kbagent supports two flow component types: `keboola.orchestrator` (classic) and `keboola.flow` (new format).

## Core concepts

- **Phase**: a named stage with `id` and `dependsOn` (list of upstream phase IDs). Phases with no `dependsOn` run first.
- **Task**: a unit of work referencing a component config, assigned to a phase via `phase` field.
- **Schedule**: stored as a `keboola.scheduler` config that targets the flow; not part of the flow config itself.

## Quick start: create a flow

```bash
# 1. See the template
kbagent flow schema

# 2. Create a simple flow from YAML
cat > flow.yaml <<'EOF'
phases:
- id: 1
name: Extract
dependsOn: []
- id: 2
name: Transform
dependsOn: [1]
tasks:
- id: 1
name: Run extractor
phase: 1
task:
mode: run
componentId: keboola.ex-db-snowflake
configId: "123456"
- id: 2
name: Run transformation
phase: 2
task:
mode: run
componentId: keboola.snowflake-transformation
configId: "789012"
EOF

kbagent --json flow new --project prod --name "Daily ETL" --file @flow.yaml
```

## List and inspect flows

```bash
# All flows across all projects
kbagent --json flow list

# Flows in one project
kbagent --json flow list --project prod

# Full phase/task breakdown
kbagent --json flow detail --project prod --flow-id 111
```

## Update a flow

```bash
# Rename only
kbagent --json flow update --project prod --flow-id 111 --name "New Name"

# Replace phases/tasks from file (validates DAG before write)
kbagent --json flow update --project prod --flow-id 111 --file @updated.yaml
```

## Schedule a flow

Schedules are stored as `keboola.scheduler` configs pointing at the flow. `flow schedule` is an upsert — if a schedule already exists for the flow it is updated in-place; otherwise a new one is created. Calling it twice with different cron expressions replaces the existing schedule.

```bash
# Daily at 06:00 UTC
kbagent --json flow schedule --project prod --flow-id 111 --cron "0 6 * * *"

# With timezone and disabled state
kbagent --json flow schedule \
--project prod --flow-id 111 \
--cron "0 8 * * 1-5" \
--timezone "Europe/Prague" \
--disabled

# Remove all schedules (idempotent)
kbagent --json flow schedule-remove --project prod --flow-id 111 --yes
```

## Delete a flow

```bash
kbagent --json flow delete --project prod --flow-id 111 --yes
```

## DAG validation

kbagent validates the phase graph client-side before every create/update:
- Unknown `dependsOn` phase IDs → `INVALID_FLOW_DAG`
- Tasks referencing unknown phase IDs → `INVALID_FLOW_DAG`
- Cycles in the phase graph → `INVALID_FLOW_DAG`

The error carries a list of human-readable violation messages.

## Component IDs

| Component | Use case |
|---|---|
| `keboola.flow` | New projects, preferred for new flows (default for `flow new`) |
| `keboola.orchestrator` | Legacy flows; most existing orchestrations use this (default for `flow detail/update/delete/schedule`) |

Both are fully supported. Use `--component-id` to override the default.
27 changes: 27 additions & 0 deletions plugins/kbagent/skills/kbagent/references/gotchas.md
Original file line number Diff line number Diff line change
Expand Up @@ -506,3 +506,30 @@ CLI hides via its four-bucket response, but they matter when interpreting result
addressing. When exporting multiple tables, each ends up in a predictable
subdirectory and there is no risk of name collisions. Override with
`--output DIR` if you need a custom location.

## Flow: default `--component-id` differs between commands

- `kbagent flow new` defaults to **`keboola.flow`** (the newer format).
- `kbagent flow detail / update / delete / schedule / schedule-remove` all
default to **`keboola.orchestrator`** (the legacy format, since most
existing flows still use it).
- Consequence: if you create a flow with `flow new` and then call
`flow detail` without `--component-id`, you will get a `NOT_FOUND` error
because kbagent looks up the ID under `keboola.orchestrator`. Always pass
`--component-id keboola.flow` when round-tripping a flow you just created
via `flow new` (or, equivalently, pass `--component-id keboola.orchestrator`
on `flow new` to keep things consistent).
- `flow list` returns both component IDs and surfaces `component_id` on each
row — use it to confirm which variant a flow lives under before issuing
detail/update/delete/schedule commands.

## Flow: `schedule` is an upsert (no `schedule-update`)

- `kbagent flow schedule` creates a `keboola.scheduler` config on first run
and **updates the existing one in-place** on subsequent runs. Running it
twice with different `--cron` values replaces the schedule — it does not
create a second one. That's why there is no separate `flow schedule-update`
command.
- To inspect or remove schedules: `kbagent flow schedule-remove` deletes all
scheduler configs that target the flow. Pair it with `--dry-run` to see the
affected configs (cron + timezone) without calling `delete_config`.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "keboola-agent-cli"
version = "0.21.1"
version = "0.22.0"
description = "AI-friendly CLI for managing Keboola projects"
readme = "README.md"
requires-python = ">=3.12"
Expand Down
10 changes: 10 additions & 0 deletions src/keboola_agent_cli/changelog.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@

# Ordered newest-first. Each value is a list of brief one-line descriptions.
CHANGELOG: dict[str, list[str]] = {
"0.22.0": [
"New: `kbagent flow list` -- list all flows (keboola.orchestrator + keboola.flow) across one or all projects; supports --project, --branch",
"New: `kbagent flow detail` -- full phase/task breakdown for a single flow config, including phase dependency graph and orphan detection",
"New: `kbagent flow schema` -- print the YAML template for flow configuration (phases + tasks) for use with --file",
"New: `kbagent flow new` -- create a flow with optional phases/tasks from a YAML/JSON --file; validates DAG before create",
"New: `kbagent flow update` -- update flow name, description, or phases/tasks; validates DAG before write; fetches current config before partial update",
"New: `kbagent flow delete` -- delete a flow config with --yes confirmation guard",
"New: `kbagent flow schedule` -- attach a cron schedule via keboola.scheduler; supports timezone and enabled/disabled state",
"New: `kbagent flow schedule-remove` -- remove all cron schedules attached to a flow; idempotent, --yes confirmation guard",
],
"0.21.1": [
"Fix: sync pull on a newly created dev branch now writes config rows (#193) -- idempotent skip guard for rows was missing a file-existence check, causing rows to be silently skipped when the branch directory was new (hash matched main because the branch is a clone)",
],
Expand Down
8 changes: 8 additions & 0 deletions src/keboola_agent_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from .commands.context import context_command
from .commands.doctor import doctor_command
from .commands.encrypt import encrypt_app
from .commands.flow import flow_app
from .commands.init import init_command
from .commands.job import job_app
from .commands.kai import kai_app
Expand All @@ -38,6 +39,7 @@
from .services.deep_lineage_service import DeepLineageService
from .services.doctor_service import DoctorService
from .services.encrypt_service import EncryptService
from .services.flow_service import FlowService
from .services.job_service import JobService
from .services.kai_service import KaiService
from .services.lineage_service import LineageService
Expand Down Expand Up @@ -83,6 +85,10 @@
app.add_typer(lineage_app, name="lineage", rich_help_panel=_BROWSE)
app.add_typer(kai_app, name="kai", rich_help_panel=_BROWSE)

# -- Flows --
_FLOWS = "Flows"
app.add_typer(flow_app, name="flow", rich_help_panel=_FLOWS)

# -- Development --
_DEV = "Development"
app.add_typer(branch_app, name="branch", rich_help_panel=_DEV)
Expand Down Expand Up @@ -192,6 +198,7 @@ def main(
sync_service = SyncService(config_store=config_store)
variables_service = VariablesService(config_store=config_store)
encrypt_service = EncryptService(config_store=config_store)
flow_service = FlowService(config_store=config_store)
workspace_service = WorkspaceService(config_store=config_store)
kai_service = KaiService(config_store=config_store)
doctor_service = DoctorService(config_store=config_store, mcp_service=mcp_service)
Expand Down Expand Up @@ -233,6 +240,7 @@ def main(
ctx.obj["sync_service"] = sync_service
ctx.obj["variables_service"] = variables_service
ctx.obj["encrypt_service"] = encrypt_service
ctx.obj["flow_service"] = flow_service
ctx.obj["workspace_service"] = workspace_service
ctx.obj["kai_service"] = kai_service
ctx.obj["doctor_service"] = doctor_service
Expand Down
32 changes: 32 additions & 0 deletions src/keboola_agent_cli/commands/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,38 @@
Use --org-id OR --project-ids (at least one required).
Token via KBC_MANAGE_API_TOKEN env var or interactive prompt.

### Flows (Orchestrator + Conditional)

kbagent flow list [--project NAME] [--branch ID]
List all flows (keboola.orchestrator + keboola.flow) across projects.

kbagent flow detail --project NAME --flow-id ID [--component-id keboola.orchestrator|keboola.flow] [--branch ID]
Show phases, tasks, and full configuration. --component-id defaults to keboola.orchestrator.

kbagent flow schema
Print the YAML format accepted by 'flow new' and 'flow update'.

kbagent flow new --project NAME --name "Name" [--component-id keboola.orchestrator|keboola.flow] [--description D] [--file YAML|@file|-] [--branch ID]
Create a new flow. --component-id defaults to keboola.flow (newer format).
--file accepts YAML with 'phases' and 'tasks' keys. DAG is validated (acyclic, refs exist).

kbagent flow update --project NAME --flow-id ID [--component-id ID] [--name N] [--description D] [--file YAML] [--branch ID]
Update a flow's name, description, or phases/tasks. --file replaces both phases and tasks.
Omitting --file leaves the flow body unchanged. DAG re-validated on write.

kbagent flow delete --project NAME --flow-id ID [--component-id ID] [--branch ID] [--yes]
Delete a flow. Does NOT remove associated keboola.scheduler configs.
Run 'flow schedule-remove' first if you want to clean up schedules.

kbagent flow schedule --project NAME --flow-id ID --cron "0 6 * * *" [--component-id ID] [--timezone TZ] [--enabled/--disabled] [--name NAME] [--branch ID]
Upsert a cron schedule: updates the existing keboola.scheduler config if one exists, creates one
otherwise. Calling twice with a new cron replaces the old schedule — no duplicates created.
Schedules are stored as Storage API configs, not a separate scheduler service.

kbagent flow schedule-remove --project NAME --flow-id ID [--component-id ID] [--branch ID] [--yes]
Remove all schedules bound to this flow (deletes all matching keboola.scheduler configs).
Idempotent: safe to run when no schedules exist.

### Development Branches

kbagent branch list [--project NAME]
Expand Down
Loading
Loading