feat(0.22.0): job run auto-resolves variableValuesId from linked variables

CLAUDE.md

@@ -256,7 +256,7 @@ kbagent config variables-clear --project NAME --component-id ID --config-id ID [
 
 kbagent job list [--project NAME] [--component-id ID] [--status STATUS] [--limit N]
 kbagent job detail --project NAME --job-id ID
-kbagent job run --project NAME --component-id ID --config-id ID [--row-id ID ...] [--wait] [--timeout N] [--branch ID]
+kbagent job run --project NAME --component-id ID --config-id ID [--row-id ID ...] [--wait] [--timeout N] [--branch ID] [--variable-values-id ID] [--no-variables]
 kbagent job terminate --project NAME (--job-id ID [--job-id ID ...] | --status any|created|waiting|processing [--component-id ID] [--config-id ID] [--branch ID] [--limit N]) [--dry-run] [--yes]
 
 kbagent storage buckets [--project NAME] [--branch ID]

plugins/kbagent/skills/kbagent/references/commands-reference.md

@@ -42,7 +42,7 @@ All commands support `--json` for structured output. Multi-project flags (`--pro
 ## Job History
 - `job list [--project NAME] [--component-id ID] [--config-id ID] [--status STATUS] [--limit N]` -- list jobs (default 50, max 500)
 - `job detail --project NAME --job-id ID` -- full job detail with timing and result message
-- `job run --project NAME --component-id ID --config-id ID [--row-id ID ...] [--wait] [--timeout N] [--branch ID]` -- run a job, optionally wait for completion (branch-aware)
+- `job run --project NAME --component-id ID --config-id ID [--row-id ID ...] [--wait] [--timeout N] [--branch ID] [--variable-values-id ID] [--no-variables]` -- run a job, optionally wait for completion (branch-aware). For configs with linked `keboola.variables` (root-level `configuration.variables_id`), kbagent auto-resolves a `variableValuesId` so transformations bind to the deployed values row. `--variable-values-id` overrides; `--no-variables` skips resolution. `NO_VARIABLE_ROWS` when the linked variables config has zero rows -- fix via `kbagent config variables-set`.
 - `job terminate --project NAME (--job-id ID [--job-id ...] | --status any|created|waiting|processing [--component-id ID] [--config-id ID] [--branch ID] [--limit N]) [--dry-run] [--yes]` -- kill running Queue API jobs. Use to stop runaway loops or clean up pile-ups from repeated `job run` calls. Two modes: by ID (single/batch) or by filter (`--status any` catches every killable state). Response partitions IDs into `killed / already_finished / not_found / failed`; safe to re-run idempotently. Kill is async -- poll `job detail` for `isFinished=true`.
 
 ## Storage

plugins/kbagent/skills/kbagent/references/gotchas.md

@@ -20,6 +20,33 @@
 - Full workflow + response shapes: see
   [variables-workflow.md](variables-workflow.md).
 
+## `job run` auto-resolves variable values (since 0.21.0)
+
+Transformations with linked `keboola.variables` used to run against empty
+strings unless the caller hand-wired a `variableValuesId` at the HTTP
+layer. `kbagent job run` now auto-resolves it: reads
+`configuration.variables_id` from the parent config (root of the
+configuration body -- same key `VariablesService` writes), picks
+`configuration.variables_values_id` if set, else the first row of the
+linked variables config.
+
+- **Override knobs**: `--variable-values-id ROW_ID` pins a specific row
+  (CI runs, what-if analysis); `--no-variables` skips resolution
+  entirely. Mutually exclusive -- passing both returns exit 2 /
+  `INVALID_ARGUMENT` before any API call.
+- **`NO_VARIABLE_ROWS`** -- the linked `keboola.variables` config exists
+  but has zero rows. Fix:
+  `kbagent config variables-set --project X --component-id C --config-id I --var KEY=VALUE`.
+- **`MALFORMED_VARIABLES_ROW`** -- Storage API returned a first row
+  without a usable `id`. Fails loud rather than silently submitting with
+  empty bindings.
+- **Empty `--variable-values-id ""`** (or whitespace) rejected at CLI
+  layer with `INVALID_ARGUMENT` -- same silent-omission class as the
+  above, caught at a different layer.
+- JSON response carries `resolvedVariableValuesId` when the resolver
+  fired, so callers verify the binding without a second `job detail`
+  round-trip.
+
 ## Sync: row deploy & manifest v3 (since 0.21.0)
 
 - `sync push` **does** deploy config rows now (previously silently skipped).
@@ -39,10 +66,10 @@
   same code. Fail-closed either way. Use
   `--allow-plaintext-on-encrypt-failure` ONLY for debugging.
 - **`keboola.variables` row secrets live in `{name, value}` list
-  elements**, not dict keys. Before 0.21.1, the encryption walker only
-  scanned `#`-prefixed dict keys and silently shipped plaintext for
-  `values: [{name: '#x', value: '...'}]`. Fixed in 0.21.1 via
-  `_is_secret_name_value_pair`. (`keboola.shared-code` rows carry
+  elements**, not dict keys. An early version of the encryption walker
+  only scanned `#`-prefixed dict keys and silently shipped plaintext for
+  `values: [{name: '#x', value: '...'}]`; fixed before 0.21.0 shipped
+  via `_is_secret_name_value_pair`. (`keboola.shared-code` rows carry
   `code_content: [string]` and have no secrets, so the walker correctly
   never fires there.) If you add a new row-hoist component with yet
   another secret shape, extend the walker -- don't patch callers.

plugins/kbagent/skills/kbagent/references/variables-workflow.md

@@ -184,3 +184,93 @@ Both are supported; pick whichever fits your workflow:
 
 They converge on the same API calls; `sync push` just routes through local YAML
 first.
+
+## Running jobs against deployed values
+
+Deploying values is only half the loop. The other half is making sure the
+Queue job actually *binds* to them at runtime. `kbagent job run` auto-resolves
+a `variableValuesId` and passes it to the Queue API -- without this, a
+transformation linked to a `keboola.variables` config runs against empty
+`{{ placeholder }}` strings (the most common silent-fail mode in pipelines
+that use variables).
+
+### End-to-end example
+
+```bash
+# 1. Deploy values
+kbagent config variables-set --project prod \
+  --component-id keboola.snowflake-transformation --config-id 15815157 \
+  --var year_start=2025 --var region=eu
+
+# 2. Run the job -- kbagent auto-resolves values row from the parent's link
+kbagent --json job run --project prod \
+  --component-id keboola.snowflake-transformation --config-id 15815157 \
+  --wait
+```
+
+Inspect the JSON output for `resolvedVariableValuesId` to verify the binding
+without a second `job detail` round-trip.
+
+### Resolution order
+
+`JobService.resolve_variable_values_id` picks, in this order:
+
+1. **Explicit `--variable-values-id ROW_ID`** -- hand-picks a values row.
+   Use for CI matrix runs ("run this job against each of our 5 environment
+   values rows") or what-if analysis.
+2. **`configuration.variables_values_id`** on the parent config -- if set,
+   this is the pin that `variables-set` and the Keboola UI both write.
+3. **First row** of the linked `keboola.variables` config -- the default-row
+   convention.
+
+The link is read from the **root** of the `configuration` body as
+`variables_id` / `variables_values_id` (snake_case). NOT nested under a
+`runtime` key -- that's a misconception an earlier draft of this feature had.
+
+### Override knobs
+
+- `--variable-values-id ROW_ID` -- pin a specific values row (overrides
+  steps 2 and 3 above).
+- `--no-variables` -- skip resolution entirely. Use for components that have
+  no linked variables config (no-op), or when you intentionally want the job
+  to run with empty bindings.
+
+They are **mutually exclusive**. Passing both returns exit 2 /
+`INVALID_ARGUMENT` before any API call.
+
+Empty or whitespace-only `--variable-values-id ""` is rejected at the CLI
+layer with the same `INVALID_ARGUMENT`; passing through as `""` would
+silently drop `variableValuesId` from the Queue body and reintroduce the
+empty-bindings silent failure.
+
+### Error codes
+
+| Code | When | Recovery |
+|---|---|---|
+| `NO_VARIABLE_ROWS` | Linked `keboola.variables` config exists but has zero rows | `kbagent config variables-set --var KEY=VALUE` |
+| `MALFORMED_VARIABLES_ROW` | Storage API returned a first row without a usable `id` | Inspect the backing config; fix or pin a specific row via `--variable-values-id` |
+| `INVALID_ARGUMENT` | Mutually-exclusive flags, empty `--variable-values-id ""` | Fix CLI args |
+
+`NO_VARIABLE_ROWS` is the common-case signal that deploy is missing: the
+parent was linked (via `sync push` or legacy UI) but the values config was
+never populated. The fix is always `variables-set` on the parent.
+
+### Response shape (`--json`)
+
+```json
+{
+  "status": "ok",
+  "data": {
+    "project_alias": "prod",
+    "job_id": "9876543210",
+    "status": "processing",
+    "resolvedVariableValuesId": "01kpn7sat48jmhvx20svaqdnf9",
+    ...
+  }
+}
+```
+
+`resolvedVariableValuesId` is present only when the resolver actually fired
+(i.e., not for `--no-variables`, and not for configs without a
+`variables_id` link). Absence of the field in the response is unambiguous
+signal that the job ran without variables bindings.

src/keboola_agent_cli/changelog.py

@@ -16,6 +16,12 @@
         "New: per-row 3-way diff -- sync status/diff now reports added/modified/deleted rows alongside parent configs; local row edits are preserved across pull",
         "New: ManifestConfigRow.metadata with pull_hash + pull_config_hash -- manifest schema bumped to v3 (v2 manifests load cleanly and upgrade in-place on next pull)",
         "Fix: _write_config_file now uses newline='' so Windows doesn't translate LF->CRLF on write, which previously caused every post-pull status to report every config as modified",
+        "New: `kbagent job run` auto-resolves `variableValuesId` for configs with linked `keboola.variables` -- transformations now run against deployed values instead of empty `{{ placeholder }}` strings (FIIA runtime loop).",
+        "New: `--variable-values-id ID` on `job run` to override the auto-resolved values row; `--no-variables` to skip resolution entirely (mutually exclusive).",
+        "New: `NO_VARIABLE_ROWS` error code when a linked variables config has zero rows (fix via `kbagent config variables-set`); `MALFORMED_VARIABLES_ROW` when the Storage API returns a first row without a usable `id` -- fail loud instead of silently submitting with empty bindings.",
+        'Reject: `--variable-values-id ""` (empty or whitespace) returns exit 2 / `INVALID_ARGUMENT` instead of silently dropping the Queue body field.',
+        "Client: `create_job` gained `variable_values_id` parameter; omitted from body when unset so existing callers retain wire-level compatibility.",
+        "Response: `kbagent --json job run` now carries `resolvedVariableValuesId` so callers can verify the binding without a second `job detail` round-trip.",
     ],
     "0.20.6": [
         "Fix: storage download-table / unload-table no longer OOM on multi-GB tables -- streamed downloads cap RAM at ~1 MiB regardless of table size (#187)",

src/keboola_agent_cli/client.py

@@ -1735,6 +1735,7 @@ def create_job(
         config_row_ids: list[str] | None = None,
         mode: str = "run",
         branch_id: int | None = None,
+        variable_values_id: str | None = None,
     ) -> dict[str, Any]:
         """Create and run a Queue API job.
 
@@ -1747,6 +1748,10 @@ def create_job(
             mode: Job mode (default: run).
             branch_id: Optional dev branch ID. When set, the job runs
                 on that branch instead of the default (production) branch.
+            variable_values_id: Optional id of a row in the linked
+                ``keboola.variables`` config. When set, the Queue API binds
+                the row's values to the job's `{{ variable }}` placeholders.
+                Omit for configurations that have no linked variables.
 
         Returns:
             Job dict from the Queue API.
@@ -1762,6 +1767,8 @@ def create_job(
             body["configData"] = config_data
         if config_row_ids:
             body["configRowIds"] = config_row_ids
+        if variable_values_id:
+            body["variableValuesId"] = variable_values_id
         response = self._queue_request("POST", "/jobs", json=body)
         return response.json()
 

src/keboola_agent_cli/commands/context.py

@@ -139,9 +139,13 @@
   kbagent job detail --project NAME --job-id ID
     Full job detail including result message and timing.
 
-  kbagent job run --project NAME --component-id ID --config-id ID [--row-id ID ...] [--wait] [--timeout N] [--branch ID]
+  kbagent job run --project NAME --component-id ID --config-id ID [--row-id ID ...] [--wait] [--timeout N] [--branch ID] [--variable-values-id ID] [--no-variables]
     Run a Queue API job. --row-id selects specific config rows (repeatable; omit to run entire config).
     --wait polls until job finishes. --timeout sets max wait in seconds (default 300). Branch-aware.
+    When the config has linked variables (configuration.variables_id), kbagent auto-resolves
+    a variableValuesId so the job binds to the deployed values row. --variable-values-id
+    overrides; --no-variables skips resolution. Error code NO_VARIABLE_ROWS when the linked
+    variables config has zero rows (run `kbagent config variables-set` to create one).
 
   kbagent job terminate --project NAME (--job-id ID [--job-id ID ...] | --status any|created|waiting|processing [--component-id ID] [--config-id ID] [--branch ID] [--limit N]) [--dry-run] [--yes]
     Kill running jobs via Queue API (POST /jobs/{id}/kill). Use to stop runaway loops or pile-ups.

src/keboola_agent_cli/commands/job.py

@@ -191,6 +191,26 @@ def job_run(
         "--branch",
         help="Dev branch ID (overrides active branch)",
     ),
+    variable_values_id: str | None = typer.Option(
+        None,
+        "--variable-values-id",
+        help=(
+            "Explicit keboola.variables values-row ID to bind. Use when the "
+            "linked variables config has multiple rows and auto-resolution "
+            "(first row) picks the wrong one. Mutually exclusive with "
+            "--no-variables."
+        ),
+    ),
+    no_variables: bool = typer.Option(
+        False,
+        "--no-variables",
+        help=(
+            "Skip variable-values resolution entirely. Use for components "
+            "that do not support variables, or when intentionally running "
+            "against empty bindings. Mutually exclusive with "
+            "--variable-values-id."
+        ),
+    ),
 ) -> None:
     """Run a job for a component configuration.
 
@@ -199,6 +219,11 @@ def job_run(
 
     When a dev branch is active (via 'branch use'), the job automatically
     runs on that branch. Use --branch to override.
+
+    When the config has linked variables (configuration.variables_id),
+    kbagent auto-resolves a variableValuesId so the job binds to the
+    deployed values row. Override with --variable-values-id or skip
+    with --no-variables.
     """
     if should_hint(ctx):
         emit_hint(
@@ -211,12 +236,35 @@ def job_run(
             wait=wait,
             timeout=timeout,
             branch=branch,
+            variable_values_id=variable_values_id,
+            no_variables=no_variables,
         )
         return
     formatter = get_formatter(ctx)
     service = get_service(ctx, "job_service")
     config_store: ConfigStore = ctx.obj["config_store"]
 
+    if variable_values_id is not None and not variable_values_id.strip():
+        formatter.error(
+            message=(
+                "--variable-values-id cannot be empty or whitespace. "
+                "Pass a row id, or omit the flag to auto-resolve the default row."
+            ),
+            error_code="INVALID_ARGUMENT",
+        )
+        raise typer.Exit(code=2)
+
+    if variable_values_id and no_variables:
+        formatter.error(
+            message=(
+                "--variable-values-id and --no-variables are mutually exclusive. "
+                "Pass --variable-values-id to bind a specific values row, or "
+                "--no-variables to skip resolution, but not both."
+            ),
+            error_code="INVALID_ARGUMENT",
+        )
+        raise typer.Exit(code=2)
+
     validate_branch_requires_project(formatter, branch, project)
     _, effective_branch = resolve_branch(config_store, formatter, project, branch)
 
@@ -230,6 +278,14 @@ def job_run(
             msg += f" [dim](waiting up to {timeout:.0f}s)[/dim]"
         msg += "..."
         formatter.console.print(msg)
+        if variable_values_id:
+            from rich.markup import escape
+
+            formatter.console.print(
+                f"[dim]Using variable values row: {escape(variable_values_id)}[/dim]"
+            )
+        elif no_variables:
+            formatter.console.print("[dim]Skipping variable-values resolution.[/dim]")
 
     try:
         result = service.run_job(
@@ -240,6 +296,8 @@ def job_run(
             wait=wait,
             timeout=timeout,
             branch_id=effective_branch,
+            variable_values_id=variable_values_id,
+            no_variables=no_variables,
         )
     except ConfigError as exc:
         formatter.error(message=exc.message, error_code="CONFIG_ERROR")

src/keboola_agent_cli/hints/definitions/job.py

@@ -75,6 +75,33 @@
         cli_command="job.run",
         description="Run a component configuration as a job",
         steps=[
+            HintStep(
+                comment=(
+                    "Resolve linked variables values row (skip when "
+                    "--no-variables; override via --variable-values-id)"
+                ),
+                client=ClientCall(
+                    method="get_config_detail",
+                    args={
+                        "component_id": "{component_id}",
+                        "config_id": "{config_id}",
+                    },
+                    result_var="detail",
+                    result_hint="dict",
+                ),
+            ),
+            HintStep(
+                comment="Look up first row of linked variables config if values_id absent",
+                client=ClientCall(
+                    method="list_config_rows",
+                    args={
+                        "component_id": '"keboola.variables"',
+                        "config_id": 'detail["configuration"]["variables_id"]',
+                    },
+                    result_var="var_rows",
+                    result_hint="list",
+                ),
+            ),
             HintStep(
                 comment="Create and submit job to Queue API",
                 client=ClientCall(
@@ -83,6 +110,7 @@
                         "component_id": "{component_id}",
                         "config_id": "{config_id}",
                         "config_row_ids": "{row_id}",
+                        "variable_values_id": 'var_rows[0]["id"] if var_rows else None',
                     },
                     result_var="job",
                     result_hint="dict",
@@ -98,6 +126,8 @@
                         "config_row_ids": "{row_id}",
                         "wait": "{wait}",
                         "timeout": "{timeout}",
+                        "variable_values_id": "{variable_values_id}",
+                        "no_variables": "{no_variables}",
                     },
                 ),
             ),
@@ -116,7 +146,18 @@
         notes=[
             "Uses the Queue API (queue.keboola.com), not Storage API.",
             "Without --wait, returns immediately after job creation.",
-            "Service layer handles both create + optional poll in one call.",
+            "Service layer handles both resolve + create + optional poll in one call.",
+            (
+                "Service auto-resolves variableValuesId from "
+                "configuration.variables_id; the client hint shows "
+                "the underlying two-request pattern."
+            ),
+            "Pass --no-variables to skip resolution entirely.",
+            (
+                "NO_VARIABLE_ROWS (exit 1) means the linked variables config "
+                "has zero rows; fix via `kbagent config variables-set` or "
+                "pass --no-variables."
+            ),
         ],
     )
 )