Skip to content

feat(flow): add flow triggers -- table triggers, not just cron (#714) - #719

Merged
padak merged 5 commits into
mainfrom
claude/issue-714-flow-triggers
Aug 30, 2026
Merged

feat(flow): add flow triggers -- table triggers, not just cron (#714)#719
padak merged 5 commits into
mainfrom
claude/issue-714-flow-triggers

Conversation

@padak

@padak padak commented Aug 29, 2026

Copy link
Copy Markdown
Member

What

kbagent could see one of the (at least) three ways a Keboola flow gets started automatically.
schedule list / schedule find / search read keboola.scheduler configs only, so a flow driven
by a table trigger — a separate Storage API resource, not a component config at all — came back
"nothing found". Twice in one investigation that was read as "this flow has no trigger", for a flow
that was already live; the wrong conclusion was that a working notification pipeline still needed
manual scheduling.

Trigger type Where it lives Before After
Cron schedule keboola.scheduler config
Table trigger Storage API /v2/storage/triggers ❌ invisible
Cross-project trigger trigger-queue config in another project ❌ invisible ⚠️ reported as not checked

Implements parts 1, 2 and 3 of the issue.

The design point: an honest negative

Cross-project triggers are reported as cross_project_triggers_checked: false, deliberately not as
an empty list
. An empty list reads as "checked, found none" — which is the exact false negative the
command exists to prevent. Detecting them means scanning every connected project and resolving each
candidate's parameters back to this project + flow; that shape isn't something I could verify, so
it is declared rather than guessed.

Part 2 landed too: schedule list / schedule find now say No cron schedules found. Table triggers and cross-project triggers were NOT checked -- see kbagent flow triggers instead of No schedules found.

Wire contract — read from the source, not guessed

Verified against Keboola's own Controller/Storage/Triggers/TriggerListAction and TriggerResponse
rather than inferred:

  • The route is declared isAvailableInBranch: false → table triggers are production-only.
    There is no branch-scoped variant to call, so --branch narrows the cron half only and
    table_triggers_branch_scoped is always false. The command never advertises a branch-scoped
    answer it cannot produce.
  • ?component= and ?configurationId= are declared filters — but whether the server applies them
    is not readable from the published source (the filter object is threaded into
    getTriggersByRequestFilter, whose body I could not locate). This codebase has been burned by
    exactly that before: the Notification Service accepts ?event= and ignores it (kbagent notification: fleet-wide audit of Flow Notification subscriptions (Notification Service API) #600). So the
    filter is sent and the result narrowed again client-side — correct whichever way the server
    behaves, and an unnarrowed response would otherwise attribute another flow's trigger to this one.
  • tables is a list of {"tableId": ...} objects, not strings; lastRun is nullable and null
    means never fired, not disabled (human mode renders it as never, not a blank cell);
    component is not necessarily keboola.flow — the API's own example is the legacy
    orchestration, so matching is on configurationId.
  • A NOT_FOUND from the endpoint degrades to an empty list, but any other error propagates
    silently swallowing a 500 as "no triggers" would recreate the very bug this fixes.

Structure

commands/flow.py is at exactly 800 code lines — the commands soft ceiling — so per CONTRIBUTING's
file-size budgets the command lives in commands/_flow_triggers.py, following the existing
_storage_snapshots.py precedent. It is mounted flat onto flow_app, so the permission key stays
flow.triggers (read) and kbagent flow --help is unchanged.

Layers: client/triggers.py (new _TriggersMixin) → FlowService.get_flow_triggers
commands/_flow_triggers.py, plus GET /flows/{project}/{config_id}/triggers over serve.

Part 3: unexplained runs get a guided next step

job detail on a keboola.flow / keboola.orchestrator job now attaches trigger_hint -- "If no
cron schedule explains this run, check kbagent flow triggers ..."
-- in JSON and over serve too,
because the consumer that was misled in #714 was an agent reading tool output, not a human at a
terminal. Human mode renders the hint and a new Created by token line: the token that created the
job (a human's email vs a scheduler's or trigger's token) is the one factual provenance signal the
Queue payload carries. Non-flow jobs are untouched.

How it was tested

  • tests/test_triggers_client.py (5) — L3 wire contract, using the controller's verbatim example
    payload: path, both query filters, numeric-id coercion, and no empty query string when unfiltered.
  • TestGetFlowTriggers in tests/test_flow_service.py (9) — table trigger surfaced, empty result
    carries the not-checked flag, another config's trigger filtered out locally, numeric
    configurationId still matches, filter sent to the API, cron half included alongside, never
    advertised as branch-scoped, 404 degrades, non-404 propagates.
  • TestFlowTriggers + TestScheduleNegativeResultWording in the CLI tests (6) — JSON shape, the
    not-checked warning in human output, never rendering, exit-code mapping, and the new
    schedule list wording.
  • The pre-existing test_human_mode_empty asserted the old No schedules found string; updated,
    since that wording is the bug.

make check exits 0 — 6385 passed, check_command_sync.py green at 268 commands.

Doc surfaces (convention #17)

This adds a CLI command, so all of them: CLAUDE.md command list, context.py AGENT_CONTEXT,
commands-reference.md, gotchas.md (tagged (since vNEXT, #714)), keboola-expert.md (new matrix
row + gotcha; prompt is 54 407 B of the 70 000 B budget), regenerated SKILL.md and
docs/web-server-endpoints.md.

No version bump and no changelog.py entry — that belongs to the release PR.

Fixes #714

padak added 4 commits August 29, 2026 07:30
kbagent could see one of the (at least) three ways a flow gets started
automatically. `schedule list` / `schedule find` / `search` read
`keboola.scheduler` configs only, so a flow driven by a TABLE TRIGGER --
a separate Storage API resource, not a component config at all -- came back
"nothing found". Twice in one investigation that was read as "this flow has
no trigger", for a flow that was already live; the wrong conclusion was that
a working notification pipeline still needed manual scheduling.

Adds `kbagent flow triggers --project P --flow-id ID`, returning cron
schedules AND table triggers in one call, plus `GET /flows/{p}/{id}/triggers`
over `serve`.

The central point is the honesty of the negative answer, so cross-project
triggers -- a trigger-queue app config in ANOTHER project -- are reported as
`cross_project_triggers_checked: false` rather than as an empty list. An empty
list reads as "checked, found none", which is the exact false negative this
command exists to prevent. Detecting them means scanning every connected
project and resolving each candidate's parameters back to this project and
flow; that is not implemented, so it is declared rather than guessed.

`schedule list` / `schedule find` now say "No cron schedules found. Table
triggers and cross-project triggers were NOT checked -- see `kbagent flow
triggers`" instead of "No schedules found".

Wire contract taken from the Storage API's own controller rather than guessed:
the route is declared `isAvailableInBranch: false`, so table triggers are
production-only and are never advertised as branch-scoped. `?configurationId=`
is sent but the result is narrowed again client-side -- the Notification
Service accepts `?event=` and ignores it (#600), and an unverified server-side
filter would silently attribute another flow's trigger to this one.

The command lives in `commands/_flow_triggers.py` because `commands/flow.py`
is exactly at the 800-code-line commands soft ceiling; it is mounted flat onto
`flow_app`, so the permission key stays `flow.triggers` and `flow --help` is
unchanged.
Independent verification against keboola/connection (TriggerRepository::
findAllByFilter, exact match, AND-ed) confirms both ?component= and
?configurationId= are applied server-side, and the server's own
testTriggersRestrictionsForReadOnlyUser E2E test confirms listing needs no
elevated privilege -- any project Storage token sees every trigger.

The docstrings and gotchas entry claimed this could not be confirmed; keep
the client-side re-narrowing (defense in depth after #600) but state the
verified facts instead of the open question.

@padak padak left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent verification of #714 / this PR

I verified the issue's claims and this PR's wire-contract assertions independently — against the keboola/connection source (commit de02f5d, src/Storage/Triggers/**, src/Controller/Storage/Triggers/**, plus its E2E suites tests/E2E/Storage/Common/TriggersTest.php and Backend/SOX/TriggersTest.php), the public keboola/storage-api-php-client (createTrigger/listTriggers/... on Client.php), and the actual keboola/component-orchestration-trigger-queue-v2 source.

Issue claim: confirmed

A full-repo grep (pre-PR) for storage/triggers, coolDown, tableTrigger, trigger-queue, kds-team.app-orchestration-trigger-queue-v2 returns zero hits — kbagent's only trigger concept really was keboola.scheduler configs. The false-negative failure mode described in #714 is real.

Wire contract: every assertion in this PR checks out

  • GET /v2/storage/triggers with ?component= / ?configurationId= — confirmed, and the server does apply both filters (TriggerRepository::findAllByFilter, exact match, AND-ed). The PR had honestly flagged this as unverifiable; I pushed 2500380 updating the docstrings/gotchas to state the confirmed fact while keeping the client-side re-narrowing as defense in depth (the #600 precedent stands).
  • Production-only — confirmed: every trigger route declares isAvailableInBranch: false, and a SOX E2E test demonstrates a dev-branch table is unreachable through this resource.
  • Response shape — confirmed 1:1 (id is a string, tables is [{tableId}] objects, lastRun nullable = never fired, component is free-form — the legacy orchestration appears in real data).
  • Auth — better than assumed: list/detail are AsReadOnlyAction, scoped only by the token's project; the server's own testTriggersRestrictionsForReadOnlyUser asserts even a read-only token sees every trigger. The stored project token is always sufficient (also now recorded in gotchas).

cross_project_triggers_checked: false is the only correct design, not a scope cut

The trigger-queue component derives the destination project at runtime by parsing the encrypted #kbcToken (parameters['#kbcToken'].split('-')[0] in its component.py), and resolves flow-vs-legacy-orchestrator by live-probing the destination project's API. The only readable hint is the optional, non-authoritative trigger_metadata.project_id UI cache. A reliable scanner cannot be built from config parameters alone, so declaring "not checked" instead of returning a false empty list is exactly right. Worth capturing in a follow-up issue if a best-effort (trigger_metadata-based, explicitly labelled heuristic) scan is ever wanted.

Live smoke test (branch build, real project)

flow triggers against a real flow on europe-west3.gcp: HTTP 200, JSON shape as documented, human mode renders the not-checked warning; schedule list prints the new "No cron schedules found. Table triggers and cross-project triggers were NOT checked" wording. (No live table trigger existed to exercise the populated path — that path is covered by the unit tests using the controller's verbatim example payload, which matches the shape asserted by connection's own E2E tests.)

Not in this PR (fine, but should not get lost)

Part 3 of #714 — the job detail / job list hint ("N runs had no matching cron schedule — check flow triggers") — is not implemented. Suggest a follow-up issue rather than growing this PR.

Verdict: correct, honestly scoped, well tested. Good to merge.

A flow run with no matching cron schedule is not necessarily manual --
table triggers and cross-project triggers also start flows and neither
appears in `schedule list`. That dead-end is exactly how the false
'this flow has no trigger' conclusion in #714 happened.

`job detail` on a keboola.flow / keboola.orchestrator job now attaches
trigger_hint (JSON and serve included -- the misled consumer in #714 was
an agent reading tool output) pointing at `kbagent flow triggers`, and
human mode renders it plus a new 'Created by token' line: the token that
created the job (a human's email vs a scheduler's or trigger's token) is
the one factual provenance signal the Queue payload carries.

Doc surfaces updated: CLAUDE.md, context.py, commands-reference.md,
gotchas.md, keboola-expert.md.
@padak

padak commented Aug 30, 2026

Copy link
Copy Markdown
Member Author

Pushed 801be8a implementing part 3 of #714 on top of the verification review above: job detail on a flow/orchestrator job now carries trigger_hint (JSON + serve included) pointing at flow triggers, and human mode adds a Created by token provenance line. Live-verified on a real flow job (hint renders in both modes; JSON key present); make check green locally -- 6384 passed. PR body updated accordingly.

@padak
padak marked this pull request as ready for review August 30, 2026 11:50
@padak
padak merged commit a31e8fd into main Aug 30, 2026
4 checks passed
@padak
padak deleted the claude/issue-714-flow-triggers branch August 30, 2026 11:50
@padak padak mentioned this pull request Aug 30, 2026
9 tasks
padak added a commit that referenced this pull request Aug 30, 2026
Batches the seven PRs merged since v0.91.0 into one version bump, one
changelog entry and one set of resolved version gates:

- #719 (#714) `flow triggers` -- table triggers, not just cron
- #717 (#711) a 401 is no longer automatically blamed on the token
- #722 (#704) setup completes in chat; skill covers setup + logout
- #718 (#716) `--conversation-id` global flag
- #706 223 stale version gates retired at the 0.80.0 floor
- #702 release process enforced rather than remembered
- #721 `get_flow_detail` docstring fix

Includes a curated What's-new entry (#717's error rework is UI-visible on
the Semantic Layer page) and the step 8-11 silent-drift review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: detect Table Triggers and cross-project triggers, not just cron schedules

1 participant