config: no CLI path to write configuration state (PUT .../state unused); `config update --set 'state...'` silently no-ops

[frantisekrehor]

Summary

kbagent can read a configuration's runtime state (config detail --with-state, v0.23.0) but has no way to write it. The Storage API exposes a dedicated PUT .../state endpoint that kbagent never calls, so the only path to seeding or resetting state is manual clicking in the KBC UI (/raw → Update State tab).

Worse, the closest-looking CLI path — config update --set 'state....=...' — silently writes to the wrong place: the value lands in configuration.state.*, the config version bumps, and the runtime state is unchanged. No error, no warning.

Why state writes are needed

State controls what an incremental component considers "already processed", so several routine operations are really state edits:

• Backfill / replay — reprocess from a chosen point after a downstream bug, without re-importing everything.
• Reset after a bad run — a component that wrote a corrupt checkpoint keeps skipping data until the state is corrected.
• Seeding a dev branch — branches start with state: {}, so any incremental component behaves as a first run there. Testing incremental behaviour at all requires seeding.
• Provisioning a new config to resume where a replaced one left off (migrations between components, project-to-project moves).
• Reproducing a customer issue by pinning the state to the exact checkpoint of a failed run.

None of these are reachable from the CLI today. The example below is just the one that forced the issue.

Concrete case: a Keboola-forced migration needs state seeding

Keboola is retiring processed_tags and query in file input mapping (changelog 2026-05-27); customers are getting support tickets with deadlines. The replacement is changed_since: adaptive.

The catch: adaptive with an empty state pulls the entire file history, not just new files. On a dev branch the state always starts empty ({}), so validating the migration before merging requires seeding the state to a known lastImportId first. Without the seed, the test run downloads years of files — on one project that meant killing a job after 118 s and thousands of files.

So the migration workflow is:

1. branch create + edit the mapping + sync push — all scriptable via kbagent ✅
2. seed the branch state to a recent lastImportId — UI only ❌
3. job run + verify the state advanced — scriptable ✅

Step 2 is the only manual step in an otherwise fully automatable, deadline-driven migration that every affected customer has to perform.

The API supports this

From keboola/storage-api-php-client/apiary.apib:

Line	Endpoint
8700	PUT /v2/storage/branch/{branch_id}/components/{component_id}/configs/{config_id}/state
9592	PUT /v2/storage/branch/{branch_id}/components/{component_id}/configs/{config_id}/rows/{row_id}/state
8851	PUT /v2/storage/components/{component_id}/configs/{config_id}/state (deprecated, no branch in URL)
9679	PUT /v2/storage/components/{component_id}/configs/{config_id}/rows/{row_id}/state (deprecated)

Body: state (required, object), max 4 MB. New code should use the branch-scoped form (pass the default branch ID for production).

Note this does not contradict the get_config_state docstring in client/configs.py — reading has no standalone resource (GET .../state returns 404/501, state is served inline in the config detail), but writing does have one. kbagent implements the read side and skips the write side.

Empirically verified today: seeding state through the UI on a dev branch, then running the job, produced exactly the expected behaviour (state advanced from the seeded lastImportId to the newest file, 24 s run instead of a full reload). The operation itself is sound — it just isn't reachable from the CLI.

Footgun: config update --set 'state...' is a silent no-op

kbagent config update --project P --component-id keboola.python-transformation-v2 \
  --config-id 25344315 \
  --set 'state.storage.input.files[0].lastImportId=176200172'

This exits 0, bumps the config version, and shows a plausible diff under --dry-run — but the runtime state is untouched. Two reasons, both in current main:

1. services/config_service.py::_resolve_configuration — set_paths are always applied to current_detail.get("configuration", {}). state is a sibling of configuration in the API response, so any path starting with state. creates configuration.state.*. Nothing special-cases it and nothing rejects it.
2. json_utils.py::set_nested_value splits on . only, so files[0] becomes a literal dict key "files[0]" — the resulting structure isn't even shaped like a state document. (List indexing only works as files.0, and only over an already-existing list.)

The result is a write that looks successful at every observable layer while doing nothing. That's arguably worth fixing independently of the feature request: config update could reject (or at least warn about) --set paths whose first segment is a known API-level sibling of configuration (state, rows, name, description, …).

Proposed surface

kbagent config state-get --project P --component-id C --config-id ID [--row-id R] [--branch ID]
kbagent config state-set --project P --component-id C --config-id ID [--row-id R] \
    --state JSON|@file|- [--branch ID] [--dry-run]

• state-get can reuse the existing get_config_state wrapper (or read state from a detail response).
• state-set calls the branch-scoped PUT .../state; --row-id routes to the row endpoint. Per the docs, when a config uses rows the root state node is unused, so row support matters for row-based components.
• Validate the payload is a JSON object and under 4 MB before the round-trip.
• --dry-run prints the current-vs-new state diff, consistent with config update.
• Permission registry: config.state-get: read, config.state-set: write.

On "state is runtime-owned"

The API docs say "the only reasonable modification of state is to delete it", and there's a fair argument that hand-editing runtime state is a footgun. Two counterpoints:

• Deletion alone (state = {}) is already a supported, documented operation — and for changed_since: adaptive it is the dangerous direction, since an empty state triggers a full reload. Seeding to a known checkpoint is the safer edit, yet it's the one with no CLI path.
• The KBC UI already exposes free-form state editing (/raw → Update State). This asks for CLI parity with an operation the platform already offers, not a new capability.
• Backfills and post-incident resets are ordinary operational work; doing them by hand in a browser doesn't make them safer, just unauditable and unrepeatable.

If the concern stands, a narrower --reset (write {}) plus a guarded --state behind explicit confirmation would still close most of the gap.

Not a duplicate

• No bulk path to fetch config details (rows + state) for many configs #197 (closed) — bulk read of rows + state; write was never in scope.
• Storage-client parity gaps & robustness improvements (from sapi-python-client comparison) #417 — mentions create_config lacking a state kwarg (low/XS); doesn't cover writing state on an existing config.
• Epic: Port keboola-mcp-server agent tools into native kbagent commands #390 — MCP tool port epic; state writing isn't among the ported tools.

Environment

kbagent v0.84.1, keboola.python-transformation-v2, Azure North Europe stack.