fix(human_gate): nest prompt_for collected values under additional_input - #237
Merged
Merged
Conversation
…text output Collected values from prompt_for were spread flat into the gate output dict, making them inaccessible at the intended nested path and allowing silent corruption of the `selected` field when the prompt_for field name was "selected".
Contributor
Author
|
@microsoft-github-policy-service agree |
…e tests
Follow-up to the prompt_for nesting fix:
* Update skill references (yaml-schema.md, authoring.md) to teach the
new `<gate>.output.additional_input.<field>` access path instead of
the old flat `<gate>.output.<field>` shape, and call out the
`context: explicit` mode limitation (nested-shorthand input
declarations are not supported — declare the parent and traverse in
Jinja2).
* Add a CHANGELOG `[Unreleased]` entry under Fixed (silent
data-corruption bug) and Changed (BREAKING template shape change)
with the migration one-liner and the explicit-mode caveat.
* Add two new tests in TestWorkflowEngineHumanGates:
- test_human_gate_web_response_nests_additional_input — stubs the
web dashboard so the web task wins the race and confirms the
second call site of `context.store` (via `_wait_for_web_gate`)
also produces the nested shape, with no flat-spread leakage.
- test_human_gate_additional_input_readable_via_template — runs a
downstream agent whose prompt template reads
`{{ ask_human.output.additional_input.answer }}` and asserts the
rendered prompt seen by the mock handler contains the value, so
the actual user-facing template contract (not just the internal
context shape) is locked in.
All 190 tests in test_gates + test_workflow + test_event_emission pass;
ruff check, ruff format --check, and ty all clean (the one pre-existing
ty warning in dialog_evaluator.py is unrelated to this PR).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #237 +/- ##
=======================================
Coverage ? 86.84%
=======================================
Files ? 64
Lines ? 10798
Branches ? 0
=======================================
Hits ? 9377
Misses ? 1421
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
fenghaitao
pushed a commit
to fenghaitao/conductor
that referenced
this pull request
Jun 4, 2026
* feat: add `type: set` step for binding computed values into context (#221) (#226)
* feat: add `type: set` step for binding computed values into context (#221)
Adds a new agent type that evaluates one or more Jinja2 expressions and
binds the results into the workflow context — no LLM call, no subprocess,
no I/O. Closes #221.
Two surface forms:
- `value:` — single expression bound as `<step>.output` (scalar / list / dict
depending on auto-detection or explicit `output_type:`).
- `values:` — named bindings rendered in a single pass against the original
pre-step context and bound as `<step>.output.<key>`.
Type detection defaults to YAML auto-parsing with a JSON-safety pass that
converts `datetime`/`date`/`time` to ISO 8601 strings and rejects any other
non-JSON-safe value so checkpoint round-trips stay stable. Empty / whitespace
renders become `""`, not `None`. Explicit `output_type:` (single-`value`
only) supports `string`, `number`, `integer`, `boolean`, `list`, `dict`.
Schema, validator, engine, and event-parity surfaces are all updated:
- `AgentDef` accepts `"set"` plus `value` / `values` / `output_type` fields,
with a model validator that rejects every irrelevant field and the
not-both/not-neither combination. Other agent types now reject set-only
fields too.
- Cross-validator walks `value` / `values.*` for template-ref checks and
rejects same-group sibling references from set templates inside parallel
groups.
- `WorkflowContext.store` now accepts any value; `_add_agent_input`
handles non-dict outputs (deep-copy for dicts, shallow for scalars/lists,
KeyError with a "is a <type>, not a dict" message when shorthand field
access targets a scalar). Trim strategies skip non-dict outputs instead
of crashing on `.items()`.
- Engine adds a dispatch branch in the main loop, parallel group, and
for-each group; emits `set_started` / `set_completed` / `set_failed`.
Output schema validation is enforced for dict outputs and rejected for
scalar outputs with a friendly suggestion.
- CLI verbose log, web `_synth_agent_or_script` synthetic replay, frontend
event-type union, `NodeType`, `NodeData`, store reducers, log builder,
and activity builder all gain set-step coverage.
- New example `examples/set-step.yaml` demonstrating single + multi
binding plus a boolean route on the derived flag.
Tests added: 161 set-related cases across schema, validator, executor,
context regression, and engine integration files. Full unit suite stays
green (2783 passed). Frontend build clean. Make lint / typecheck clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address PR review findings on set step (#221)
Addresses the comprehensive PR review on `feature/issue-221-set-step`.
Behavioural fixes:
- Parallel and for-each `set` branches now emit `set_started` /
`set_completed` / `set_failed` and apply `output:` schema validation,
matching the linear main-loop dispatch. Previously the dashboard
silently missed set lifecycle events inside groups and the schema
contract was only enforced in linear position. Extracted the shared
logic into `WorkflowEngine._run_set_step`.
- `_to_json_safe` now raises `ExecutionError` for non-string dict keys
instead of silently `str()`-coercing them. Silent coercion risked
collisions like `{1: "a", "1": "b"}` losing an entry, and contradicted
the rest of `_to_json_safe`'s "JSON-safe or error" contract.
- `_coerce`'s `auto` branch now logs at debug level when YAML parsing
fails, so a malformed render that demotes to a string is observable.
- `_trim_truncate` and `_trim_summarize` log at debug level when they
skip / render non-dict outputs so a user debugging "context still too
big" can see why.
- `_trim_summarize` now uses `json.dumps` instead of `repr` for non-dict
summaries, matching the rest of the set pipeline.
- Boolean coercion error message now includes a "rendered empty"
hint when the template produced whitespace-only output.
- Removed unreachable `TypeError` from `_coerce`'s `int()`/`float()`
except clauses (the input is always a `str`).
API tightening:
- `SetOutput.output_type` and `SetCompletedData.output_type` (frontend)
are now narrowed from `str` to the `SetOutputType` literal union
`auto | string | number | integer | boolean | list | dict`.
- Extracted `render_set_value_repr` + `SET_VALUE_REPR_MAX` into
`set_step.py` and reused them from `WorkflowEngine` and the web
server's synthetic-replay branch. Removes a duplicated 512-char
truncation literal and a duplicated try/except `json.dumps`/`repr`
fallback, and adds an `logger.error` on the fallback path so corrupt
checkpoints are visible rather than silently downgraded.
- Synthetic replay now propagates the declared `output_type` from the
agent def instead of hard-coding `"auto"` so a resumed dashboard
matches the live emitter.
- Moved `import copy` to module top in `context.py` (was deferred
inside `_add_agent_input`).
- Added `logger` to `context.py` for the new debug paths.
Documentation:
- Set-step module docstring now accurately describes the type-detection
contract: `_to_json_safe` raises on unknown Python objects (not
"falls back to string") and converts `datetime`/`date`/`time` to
ISO-8601 strings.
- Fixed misleading "URL-like string" rationale in the auto-detect
None-fallback to correctly describe pure-comment renders.
- Fixed self-contradictory "sentinel" framing in `_add_agent_input` —
the seed value is just the natural default shape for each output type.
- Dropped trailing-edge `(issue #221)` parenthetical from the dispatch
comment and `(per-key typing may be added in a future enhancement)`
speculation from the schema docstring.
- Trimmed stale "cheap but reusable" rationale on `_YAML_LOADER`.
- Pointed the "guaranteed by the schema validator" comment at the
actual function name (`AgentDef.validate_agent_type`).
Tests added / tightened:
- Direct `_coerce` auto branch: tightened over-loose YAML parse failure
test (now asserts the raw string is returned verbatim); added explicit
pure-comment-render and null-marker coverage.
- End-to-end execute() tests for date-like renders normalising to ISO
strings (single + multi).
- Dedicated `TestRenderSetValueRepr` covering the 512-char truncation
boundary with the shared helper.
- `set_failed` event assertion on both schema-validation failure paths
(scalar-with-schema, multi-with-mismatch), in addition to the existing
template-error case.
- New `TestSetInParallelGroup.test_set_in_parallel_emits_set_events`
and `test_set_in_parallel_output_schema_enforced` lock in the
group-parity fixes.
- New `TestSetInForEach.test_set_in_for_each_emits_set_events_per_item`
asserts per-iteration `set_started`.
- New `TestSyntheticReplaySetStep` covers the web `_synth_agent_or_script`
set branch for scalar, dict, and declared-output-type cases, plus
the shared-helper invariant.
- Renamed `TestSetCheckpointRoundtrip` →
`TestSetContextSerialization` (the test exercises
`WorkflowContext.to_dict`/`from_dict` directly; engine resume is
covered separately by the CLI suite).
All 2798 unit tests pass; lint clean; frontend builds clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: document `type: set` step across README, syntax docs, and conductor skill (#221)
User-facing documentation now covers the new set step type added by
the engine in this PR. Each surface follows the conventions already
established by the script-step and sub-workflow docs.
- **README.md**: add "Set steps" to the Features list and a row for
`set-step.yaml` in the Examples table.
- **examples/README.md**: new "Set Step Examples" section with both
severity branches shown as runnable commands.
- **docs/workflow-syntax.md**: full Set Steps section after Script
Steps — schema, type-detection table, multi-binding ordering rule,
routing semantics for dict vs scalar outputs, output-schema rules,
composition with parallel / for-each, restrictions, and event
contract.
- **plugins/conductor/skills/conductor/SKILL.md**: new `type: set`
row in the Key Concepts table.
- **plugins/conductor/skills/conductor/references/yaml-schema.md**:
add set-only fields (`value`, `values`, `output_type`) to the
AgentDef block, a Set agent restrictions paragraph, and a new
"Set Agent Schema" section covering type detection, routing,
composition, and event payload shape.
- **plugins/conductor/skills/conductor/references/authoring.md**:
full "Set Steps" how-to after Script Steps mirroring the
workflow-syntax depth.
- **AGENTS.md**: extend the Workflow Execution Flow with a step
for set dispatch (including the shared `_run_set_step` helper),
and add a Key Patterns bullet covering set typing, `_to_json_safe`,
`WorkflowContext.store` widening, and `_add_agent_input`
non-dict semantics.
No code changes. `make lint` clean; `make validate-examples` clean;
targeted `pytest -k 'set or synthetic'` still 184 passing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(web): add dedicated SetNode + SetDetail components for type: set (#221)
The earlier commits in this PR plumbed set_started/set_completed/
set_failed events end-to-end (engine → web server → frontend types →
store reducers) and added set_output_type / set_output_keys /
set_value_repr fields to NodeData. However, two visual surfaces still
fell through to the default LLM-agent rendering:
- graph-layout.ts only mapped script/human_gate/workflow to specific
flow node types — set steps rendered as AgentNode (chat-bubble icon,
empty model area, no value preview).
- DetailPanel's switch dispatched to ScriptDetail/GateDetail/etc with
AgentDetail as default — set steps opened AgentDetail, which expects
output/model/tokens/cost_usd/iterationHistory and ignored the set-
specific fields the reducer carefully populated.
This commit adds the missing visual treatment:
- SetNode.tsx: graph node modelled on ScriptNode but with a Variable
lucide icon and a set-specific stats line — key count for multi-
binding outputs, value preview for single-binding scalars, plus
elapsed time. Uses the same status colours, transition animations,
and live-elapsed timer as the other node types.
- SetDetail.tsx: detail panel with a "Set" status badge, metadata
grid (Elapsed, Output Type, Bindings), and an OutputViewer that
surfaces value_repr (already JSON-truncated server-side at ~512
chars) with the standard expand/copy controls.
- WorkflowGraph: register setNode in the nodeTypes map.
- graph-layout: add `nodeType === 'set'` → `setNode` mapping.
- DetailPanel: add `case 'set': return SetDetail` to the dispatch
switch.
Verified via `npm run build` (2089 modules, +2), `make lint` clean,
`pytest -k 'set or synthetic'` still 184 passing, and `conductor run
examples/set-step.yaml` produces the expected output.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(engine): add type: wait step for in-process pauses (#224)
* feat(engine): add type: wait step for in-process pauses (#218)
A new step type that pauses workflow execution for a parsed duration
via asyncio.sleep. Cross-platform, no shell dependency. Composes with
routing loop-backs for polling, rate-limit cooldowns, and demos.
YAML:
agents:
- name: cooldown
type: wait
duration: 60s # int/float seconds, "Ns/Nm/Nh/Nms", or Jinja2
reason: "Cooling down" # optional, shown in dashboard
routes:
- to: next_step
Behavior:
- Duration accepts plain numbers, suffixed strings (ms/s/m/h), and
Jinja2 templates. Workflow.input.* is available without an explicit
input: declaration (wait joins script/workflow in
_LOCAL_RENDER_AGENT_TYPES).
- Schema enforces 0 < duration <= 24h and rejects boolean durations
pre-coercion via a mode="before" field validator (Pydantic v2 would
otherwise accept True as int 1).
- Sleep races against the engine's interrupt_event so Esc/Ctrl+G
cancels in-flight waits immediately. The event is NOT cleared by
the executor — the engine's between-step _check_interrupt consumes
it so the user still gets the normal interrupt menu.
- Workflow-level limits.timeout_seconds cancels long waits via
LimitEnforcer.wait_for_with_timeout (same wrapping used by scripts).
- Wait counts toward limits.max_iterations (step counter) but is not
subject to max_agent_iterations (per-LLM-agent tool counter — N/A).
- Public output contract is strictly {"waited_seconds": float} per
the issue spec. Extra metadata (requested_seconds, reason,
interrupted) lives in event payloads only.
Events:
- The existing generic agent_started fires before type dispatch with
agent_type: "wait", so dashboards keyed on agent lifecycle pick it
up. Type-specific wait_started / wait_completed / wait_failed
carry the additional fields (mirrors the script convention).
- Resume replay extends _synth_agent_or_script with a wait branch
so checkpointed wait steps round-trip cleanly.
Validation:
- Rejects wait inside parallel groups and as for_each inline agents
(consistent with script).
- Forbids 22 incompatible fields on wait (prompt, model, command,
args, env, working_dir, tools, options, workflow, retry, dialog,
reasoning, timeout, timeout_seconds, max_session_seconds,
max_agent_iterations, max_depth, input_mapping, system_prompt,
provider, output).
- Forbids duration/reason on all non-wait types.
Dashboard:
- New 'wait' NodeType with WaitNode (Clock icon) and WaitDetail
component. Graph layout maps wait -> waitNode. Workflow store
carries duration_seconds, waited_seconds, requested_seconds,
reason, and interrupted on NodeData. Activity log renders
"Waiting Xs — reason" and "Wait completed (Xs) — interrupted".
Documentation:
- New examples/wait-step.yaml demonstrates a polling pattern with
templated poll interval and route loop-back.
Tests:
- tests/test_engine/test_duration.py: pure parser (26 cases).
- tests/test_config/test_wait_schema.py: schema validation (41 cases:
valid forms, required duration, bool rejection, bounds, forbidden
fields, parallel/for_each rejection, wait-only fields on other
types).
- tests/test_executor/test_wait.py: sleep accuracy, interrupt
cancellation (verifies event stays set for engine consumption),
templated durations, runtime validation (11 cases).
- tests/test_engine/test_wait_workflow.py: end-to-end (6 cases) —
linear wait, strict output contract, workflow timeout cancellation,
event emission shape, templated duration from workflow input,
interrupt-event cancels in-flight sleep.
Closes #218
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore(wait): address PR #224 review findings
Follow-up to feat(engine): add type: wait step (#218) addressing
findings from the pr-review-toolkit pass.
Code fixes:
- schema.py: drop the duplicate isinstance(value, bool) guard inside
_validate_wait_duration. The reject_bool_duration field validator
(mode="before") already catches booleans pre-coercion in normal
construction; the duplicate was unreachable. Replace with a docstring
note pointing at the field validator. (Flagged by dead-code-finder,
type-design-analyzer, and comment-analyzer.)
- engine/workflow.py wait dispatch preview block:
* Replace the redundant `except (ValueError, Exception)` tuple with a
bare `except Exception` (ValueError is a subclass).
* Add a `logger.debug` line on preview render failure so the path is
no longer silent.
* For preview_reason, fall back to None instead of the raw template
string — the dashboard previously displayed literal Jinja markup
like `{{ x }}` until wait_failed fired; None is the correct "absent"
signal. (Flagged by silent-failure-hunter and code-reviewer.)
- executor/wait.py _sleep_with_interrupt cleanup: narrow
`contextlib.suppress(CancelledError, Exception)` to `suppress(CancelledError)`
to match the success-path cleanup. Genuine errors from future, non-
trivial awaitables should not be silently swallowed during cleanup.
(Flagged by silent-failure-hunter.)
Test additions / improvements:
- test_web/test_server.py: new test_emits_wait_events_for_wait_type
pins the _synth_agent_or_script wait branch contract (event names,
synthetic flag, duration_seconds/reason/interrupted fields). This
was the largest coverage gap — resume replay had no test.
- test_engine/test_wait_workflow.py: new
test_emits_wait_failed_on_runtime_validation verifies a wait_failed
event is emitted with error_type="ValidationError" and the expected
message before the exception unwinds. Closes the gap where a
refactor dropping the try/except in the dispatch branch would leave
the dashboard with a hanging "started but never completed" node.
- test_executor/test_wait.py + test_engine/test_wait_workflow.py:
loosen wall-clock lower bounds (>= 0.09 / >= 0.04 / >= 0.01 / etc.)
that risked CI flakes on loaded runners. The executor's contract is
"sleep at least roughly this long unless interrupted", and the
interrupted=False check is the real invariant — the precise
elapsed value is not what these tests are pinning.
All 1463 tests in tests/test_config + tests/test_engine + tests/test_executor
+ tests/test_web pass. `make lint` is clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(wait): document type: wait step across user docs and skill references
Follow-up to PR #224 — propagate the new `type: wait` step into all
user-facing documentation and the bundled conductor skill so authors
and AI assistants discover it.
User docs:
- docs/workflow-syntax.md: add full Wait Steps section after Script
Steps (duration formats, output contract, polling loop-back pattern,
cancellation semantics, restrictions, example link). Update the
type literal comment to include `wait`. Note that wait steps count
toward `max_iterations`. Update the dialog-mode forbidden-type list.
- docs/configuration.md: add `wait` to the list of agent types where
`reasoning.effort` is rejected (don't call a model).
- README.md: add wait-step.yaml to the examples table.
- AGENTS.md: document `executor/wait.py` and the new top-level
`conductor/duration.py` helper module. Update the workflow execution
flow steps to include wait.
- examples/README.md: add a "Step Types" section documenting both
script-step.yaml (previously undocumented) and the new
wait-step.yaml.
Skill references (plugins/conductor/skills/conductor/):
- SKILL.md: add wait entry to the Key Concepts table.
- references/yaml-schema.md: extend the type-literal enumeration; add
a full Wait Agent Schema section (duration format, output, full
forbidden-field list, cancellation); update parallel-group and
for-each validation rules to mention wait; extend the
reasoning/retry/timeout_seconds forbidden-type comments.
- references/authoring.md: extend the type-literal comment; add a
full Wait Steps (`type: wait`) section (duration format, output,
polling loop-back pattern, cancellation, restrictions); extend the
reasoning/retry/timeout_seconds/dialog forbidden-type lists.
CHANGELOG.md:
- Add an Unreleased entry summarizing the wait step feature, with
PR/issue references.
Verification:
- `make validate-examples` passes (including wait-step.yaml).
- All cross-references to step-type restrictions are now consistent
across user docs and skill references.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* examples(wait): add wait-smoke.yaml + fix wait-step env-templating bug
Two related changes to the wait-step examples:
1. Add `examples/wait-smoke.yaml` — a minimal wait-only workflow with
three sequential wait steps (suffixed string, templated, plain
numeric) and no LLM / no script / no provider dependency. Useful as
a smoke test for installation, dashboard rendering, and exercising
the interrupt / workflow-timeout cancellation paths. Verified end-
to-end with `conductor run` and `--web-bg`.
2. Fix `examples/wait-step.yaml` — the polling example tried to read
`max_attempts` from the script's `env:` value via a Jinja template
(`READY_AFTER: "{{ workflow.input.max_attempts }}"`), but the docs
in `docs/workflow-syntax.md` explicitly state that `env` values are
passed as-is to the subprocess and NOT Jinja-rendered. The
subprocess then crashed with
`ValueError: invalid literal for int() with base 10: '{{ workflow.input.max_attempts }}'`,
which in turn made the script produce no JSON output and the route
condition `when: "status == 'ready'"` fail with
`'status' is not defined`. Fix: pass the threshold as a positional
`args` entry (which IS Jinja-templated) and read it via `sys.argv[1]`
in the inline Python. Added an inline comment in the example
pointing at the args-vs-env templating distinction so future
readers don't repeat the mistake. Verified end-to-end with
`--input poll_interval_seconds=2 --input max_attempts=2` —
completes cleanly with `result: "...readiness on the second attempt."`.
Also updated:
- `README.md` examples table — added `wait-smoke.yaml` row.
- `examples/README.md` — added a wait-smoke.yaml entry under the
Step Types section with usage examples (including `--web` for
dashboard inspection and the high-`middle_duration_ms` form that
triggers the workflow-timeout cancellation path).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(engine): add `type: terminate` step (#219)
Closes #219.
Adds a new `type: terminate` workflow step that ends execution with an explicit `status` (`success` | `failed`) and a structured `reason`, distinguishable from the default `$end` path in the CLI exit code, dashboard state, and event logs.
- Schema + validator: new step type with `status` / `reason` / `output_template` fields, forbidden on other types; not allowed inside parallel-group members or for_each inline agents; treated as a path-enumeration sink.
- Engine: dispatch branch with deferred-raise pattern that keeps the dashboard alive for explicit terminations (parity with successful runs); new `WorkflowTerminated` exception with structured payload; `SubworkflowTerminatedError` boundary downgrade so a child's failed terminate is treated as a normal sub-workflow failure (parent's `workflow_failed` does NOT inherit `is_explicit: true`); child's rendered output / reason / step name preserved as attributes on the wrapper.
- CLI: `run` and `resume` print rendered output JSON to stdout, surface the structured reason on stderr, exit non-zero on failed termination; resume hints suppressed.
- Dashboard: distinct `TerminateNode` (octagon icon, status-themed border, `TERMINATE · SUCCESS` / `TERMINATE · FAILED` sub-label, inline reason); workflow-level "Workflow Terminated" banners in green (success) and red (failed) with reason + `terminated_by` line.
- Tests: 46+ new tests across schema, validator, engine, sub-workflow, CLI, exception layers; full suite green.
- Docs: README, AGENTS.md, CHANGELOG, docs/workflow-syntax.md, examples/README.md, conductor skill references (SKILL.md, authoring.md, yaml-schema.md), and the schema.py class docstring all updated.
- Example: `examples/terminate.yaml` demonstrates success / failed / pass-through paths.
Smoke-tested live via Playwright on both terminate paths against the real dashboard.
🤖 Co-authored with [Copilot CLI](https://github.com/github/copilot-cli).
* feat(providers): structured runtime.provider config for Copilot custom endpoints (#225)
Closes #136.
`runtime.provider` now accepts either the bare string shorthand
(`provider: copilot`) or a structured `ProviderSettings` object that
forwards a `ProviderConfig` to the Copilot SDK's
`create_session(provider=...)` parameter. This lets workflows route the
Copilot SDK at OpenAI-compatible / Azure / Anthropic endpoints (Ollama,
vLLM, LM Studio, Azure OpenAI, etc.) instead of being locked to the
GitHub Copilot service.
Schema (`src/conductor/config/schema.py`):
- New `ProviderSettings` model (frozen) with `name` / `type` /
`wire_api` / `base_url` / `api_key` / `bearer_token` / `headers` /
`azure`. `frozen=True` avoids the Pydantic gotcha where
cross-field `model_validator` invariants don't re-fire on
per-attribute assignment.
- `api_key` and `bearer_token` are `SecretStr` (redacted in
`model_dump`, dashboard payloads, event logs).
- New `AzureProviderOptions` mirrors the SDK's nested
`azure.api_version` shape.
- `RuntimeConfig.provider` is `ProviderSettings` with a `mode='before'`
validator that coerces strings, plus `validate_assignment=True` so
CLI mutation still revalidates.
- Strict validators reject anchorless / empty / broken combinations
that would silently no-op at the SDK boundary: `wire_api` / `type` /
`headers` / `azure` cannot stand alone without an endpoint anchor
(`base_url` / `api_key` / `bearer_token`); empty `headers`, empty
`SecretStr`, and `azure: {api_version: null}` are rejected. Non-
copilot `name` plus any structured field is rejected (structured
config for Claude / openai-agents is out of scope here).
- `model_serializer` collapses `ProviderSettings` back to a bare string
when only `name` is set so serialized YAML/JSON stays byte-compatible
with the prior schema.
Copilot provider (`src/conductor/providers/copilot.py`):
- New `provider_settings` kwarg on `__init__`. Tracks
`_default_model_explicit` so the custom-routing warning fires based
on whether the caller supplied a model, not on a fragile string
comparison against the SDK fallback.
- `_resolve_sdk_provider_config()` builds the SDK `ProviderConfig` dict
with env-var fallbacks: `COPILOT_PROVIDER_BASE_URL` →
`OPENAI_BASE_URL` for `base_url`; `COPILOT_PROVIDER_API_KEY` (only)
for `api_key`; `COPILOT_PROVIDER_BEARER_TOKEN` (only) for
`bearer_token`. Ambient `OPENAI_API_KEY` is intentionally NOT a
fallback — that would silently leak an OpenAI credential to whatever
`base_url` points at. Activation is gated on `has_custom_routing()`
so ambient env vars alone never divert default Copilot traffic.
- `_apply_provider_config()` mutates session kwargs in-place; called
from both the agent `_execute_sdk_call` path and the dialog
`execute_dialog_turn` path so all sessions hit the same endpoint.
- Raises `ProviderError` when custom routing activated but every
resolved field is falsy (silent no-op was previously possible if all
expected env vars were missing).
- Warns when `api_key` and `bearer_token` BOTH resolve (from any
YAML × env combination) since the SDK silently prefers
`bearer_token`; warns when custom routing is active without an
explicit `runtime.default_model`.
Plumbing:
- `providers/factory.py` `create_provider` accepts `provider_settings`
and forwards to `CopilotProvider` only.
- `providers/registry.py` passes `runtime.provider` to the factory only
when the resolved provider type matches the settings' `name`;
switches all bare-string reads to `.name`.
- `cli/run.py` reads `.name` for logging, adds a redacted
`_describe_provider()` helper used in verbose output, and emits a
notice when `--provider` override discards structured YAML fields
(wired into both `run` and `resume` for parity).
Docs & example:
- `examples/copilot-local-llm.yaml` — Ollama + Azure OpenAI examples.
- `AGENTS.md` — new bullet under "Key Patterns" documenting the object
form, env-var fallbacks, and CLI override semantics.
Tests:
- `tests/test_config/test_provider_settings.py` (new) — coercion,
validators (including the new anchorless / empty / broken-combo
rejection cases), serialization, `has_custom_routing` gate.
- `tests/test_providers/test_copilot_provider_routing.py` (new) —
resolver behavior (env precedence, secret unwrap, ambient
`OPENAI_API_KEY` non-fallback regression, YAML×env dual-credential
warning, raise-on-empty regression, default-model warning suppressed
when caller supplies a model) and `create_session` plumbing for both
agent and dialog paths plus registry forwarding.
- Updated assertion sites across the test tree to read
`runtime.provider.name` now that the field is a model.
Follow-ups (called out for later):
- Structured provider config for `claude` / `openai-agents`.
- Persisting a redacted provider fingerprint in checkpoints so resume
can warn on configuration drift.
- Per-field CLI overrides (e.g. `--provider-base-url`).
- Converting `ProviderSettings` to a discriminated union on `name` —
would eliminate the cross-field validator in favor of structural
per-branch typing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: external workflow friction - minimal evidence-anchored fixes (#232)
* fix(parser): use greedy fence regex for JSON with embedded backticks
parse_json_output (executor/output.py) and _extract_json
(providers/copilot.py) both used a non-greedy fenced-block regex:
r'`(?:json)?\s*\n?(.*?)\n?`'
When an agent emitted JSON whose string values contained triple-backtick
substrings (common for Markdown- or shell-snippet-bearing payloads), the
first inner ` closed the match prematurely. The extracted substring was
invalid JSON, triggering parse-recovery loops and token waste -- the
exact failure mode reported in the external-workflow-friction
brainstorm Issue #1.
Fix: switch both call sites to a greedy capture with re.DOTALL:
r'`(?:json)?\s*\n(.*)\n`'
With DOTALL the greedy '.*' terminates at the last triple-backtick in
the string, which is the correct boundary. The existing first-{/[
heuristic and brace-match fallback remain as final attempts; the
canonical json.loads call at the end is the unchanged failure point
for genuinely malformed JSON.
Test: tests/test_executor/test_output.py gains
test_parse_json_with_triple_backticks_inside_string which fails on main
and passes after this commit. The malformed-input test is retained as a
regression guard for the unchanged error message.
Plan: docs/projects/usability-features/external-workflow-friction.plan.md
(Item 1, FR-1/FR-2/FR-3).
* fix(cli): abort --web-bg before fork when workflow has human_gate
conductor run --web-bg and conductor resume --web-bg forked a detached
background process to run the workflow. When the workflow reached a
human_gate step, Prompt.ask() read from the now-closed stdin and the
child crashed with EOFError. The parent only saw 'Background process
exited immediately with code 1' -- nothing pointed at the actual
incompatibility, --skip-gates, or the foreground --web alternative.
Fix: add _abort_web_bg_if_human_gate() helper in cli/app.py that loads
the workflow, walks config.agents, and if any 'type: human_gate' is
present (and --skip-gates is not set) aborts with the four-option
guidance message before launch_background() forks anything. Call site
added to both 'run' and 'resume' per the run/resume parity rule in
AGENTS.md.
Tests: tests/test_cli/test_web_flags.py gains a TestWebBgHumanGateValidation
class with three CliRunner tests:
- test_web_bg_with_human_gate_aborts_before_fork
- test_web_bg_with_human_gate_and_skip_gates_proceeds
- test_resume_web_bg_with_human_gate_aborts_before_fork
All three mock launch_background, run in-process, and produce no
subprocess. Deterministic.
Plan: docs/projects/usability-features/external-workflow-friction.plan.md
(Item 4, FR-6).
* docs: omit-output guidance, --web-bg gate constraint, brainstorm + plan
Three documentation updates plus the planning artifacts behind this
fix cluster.
1. docs/workflow-syntax.md: new 'Choosing whether to declare output:'
subsection. Explains that declaring 'output:' both injects a JSON
schema instruction AND parses the response, and that omitting it
lets downstream agents read '<agent>.output.result' for prose or
large/nested JSON. Closes the documentation gap behind
external-workflow-friction Issue #2.
2. docs/cli-reference.md: --web-bg section now documents the
human_gate incompatibility and the four supported options matching
the new pre-fork validation in cli/app.py.
3. CHANGELOG.md [Unreleased]: entries for the fence regex fix, the
--web-bg human_gate validation, and the docs additions.
4. docs/projects/usability-features/: brainstorm and plan documents
that produced this fix cluster. The plan is the source of truth for
which brainstorm items were scoped in (G1/G3/G4 -- the three
commits in this branch) versus explicitly deferred (NG1-NG7 in
the plan, with thresholds to reopen). Kept in-tree so future
contributors can answer 'why was item X not done' without spelunking
PR history.
* fix(cli): emit --web-bg human_gate abort via plain echo + tolerate stderr in test
Click 8.3+ separates result.output (stdout) from result.stderr, and Typer renders BadParameter as a Rich panel that word-wraps narrow in CI runners, splitting '--skip-gates' across box-border lines so the literal substring is absent from the captured output. Switch the abort path to plain typer.echo(err=True) + typer.Exit so the message renders verbatim, and update the asserting tests to read result.stderr too.
* fix: address PR #232 review — multi-fence parse, for_each gate, resume gate
Addresses inline review feedback from jrob5756 on PR #232 (all flagged as
non-blocking but worth fixing):
1. executor/output.py + providers/copilot.py — multi-fence regression:
The greedy regex from the original PR fixed backticks-in-string but
broke responses with multiple fenced JSON blocks (the greedy capture
spans from the first '\\\' to the last, swallowing everything in
between and failing json.loads with 'Extra data').
New two-stage strategy: try non-greedy re.findall + per-candidate
try-parse first (first valid wins → multi-block-safe), fall back to
the greedy single capture (recovers backticks-in-string).
Adds regression tests in both test_output.py and test_copilot.py
that pin 'first valid block wins' on the multi-block input from the
reviewer's repro.
2. cli/app.py _abort_web_bg_if_human_gate — for_each coverage gap:
The check only walked config.agents. A workflow with human_gate
declared inline inside a for_each.agent passed the guard and still
silent-crashed under --web-bg. Now also walks config.for_each and
inspects each group's inline agent. Regression test added.
3. cli/app.py resume command — checkpoint-only resume gap:
When the user ran 'conductor resume --from <ckpt> --web-bg' without
a workflow argument, resolved_workflow was None and the gate guard
was skipped entirely. Now reads workflow_path from the checkpoint
JSON and runs the same check. Regression test added.
Tests: 959 passed, 3 skipped (was 954 before this commit + 5 new).
Lint: ruff check + format clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Lucio Cunha Tinoco <lucioti@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: release 0.1.18 (#233)
Bump version to 0.1.18 and finalize CHANGELOG.
Adds previously undocumented entries for:
- type: set step (#226, closes #221)
- silent-aware _verbose_console (#223, closes #209)
Plus existing Unreleased entries for type: wait (#224), type: terminate
(#219), structured runtime.provider (#225), and external-workflow-friction
fixes (#232).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(providers): add claude-agent-sdk provider (#104)
* feat(providers): add claude-agent-sdk provider
Add a new provider that uses the claude-agent-sdk package to delegate
the agentic loop, tool execution, and structured output extraction to
the Claude Code CLI. Unlike the raw claude provider, this provider does
not manage its own retry logic, MCP servers, or tool wiring — these are
handled by the SDK runtime.
- New provider: ClaudeAgentSdkProvider with execute(), validate_connection(), close()
- Message dispatch via type(message).__name__ matching real SDK class names
- Tool result pairing: tracks pending tool_use IDs to emit agent_tool_complete
with actual results from ToolResultBlock (not fake nulls)
- Structured output via ClaudeAgentOptions.output_format (json_schema)
- Event callback parity: agent_turn_start, agent_message, agent_reasoning,
agent_tool_start, agent_tool_complete
- 22 tests using real SDK types (AssistantMessage, TextBlock, etc.)
- Updated schema, factory, __init__, docs, and example workflow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(claude-agent-sdk): enhance logging with verbose and full mode options
* fix: post-merge CI fixes for type-check and install-scripts
- providers/registry.py: add 'claude-agent-sdk' to ProviderType Literal
so _get_provider_type_for_agent's return type covers the new provider
- providers/claude_agent_sdk.py: type the missing-extra fallbacks as
Any (ty was treating None-typed query/ClaudeAgentOptions as the only
shape) and cast SDK message objects to Any in dispatch branches where
hasattr/getattr narrowing isn't enough for ty
- tests/test_providers/test_claude_agent_sdk.py: pytest.importorskip
the claude_agent_sdk module so collection succeeds in the base
install (install-scripts CI job runs without the optional extra)
* fix(typecheck): ty:ignore unresolved-import for optional claude_agent_sdk
The Type Check CI job runs uv sync --group dev without optional extras,
so the upstream claude_agent_sdk package isn't installed. The try/except
ImportError pattern handles this at runtime, but ty's static resolver
still complains. Add a ty:ignore directive on the import line.
Verified locally with the package uninstalled: 0 errors from ty.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jrob5756 <jasont.robert@gmail.com>
* fix(human_gate): nest prompt_for collected values under additional_input (#237)
* fix(human_gate): nest prompt_for values under additional_input in context output
Collected values from prompt_for were spread flat into the gate output dict,
making them inaccessible at the intended nested path and allowing silent
corruption of the `selected` field when the prompt_for field name was
"selected".
* fix(context): support nested explicit input projection
* Revert "fix(context): support nested explicit input projection"
This reverts commit 3d95cba7813f5200744d136ff633e4a6ece7f0d2.
* docs(human_gate): document additional_input nesting + add web/template tests
Follow-up to the prompt_for nesting fix:
* Update skill references (yaml-schema.md, authoring.md) to teach the
new `<gate>.output.additional_input.<field>` access path instead of
the old flat `<gate>.output.<field>` shape, and call out the
`context: explicit` mode limitation (nested-shorthand input
declarations are not supported — declare the parent and traverse in
Jinja2).
* Add a CHANGELOG `[Unreleased]` entry under Fixed (silent
data-corruption bug) and Changed (BREAKING template shape change)
with the migration one-liner and the explicit-mode caveat.
* Add two new tests in TestWorkflowEngineHumanGates:
- test_human_gate_web_response_nests_additional_input — stubs the
web dashboard so the web task wins the race and confirms the
second call site of `context.store` (via `_wait_for_web_gate`)
also produces the nested shape, with no flat-spread leakage.
- test_human_gate_additional_input_readable_via_template — runs a
downstream agent whose prompt template reads
`{{ ask_human.output.additional_input.answer }}` and asserts the
rendered prompt seen by the mock handler contains the value, so
the actual user-facing template contract (not just the internal
context shape) is locked in.
All 190 tests in test_gates + test_workflow + test_event_emission pass;
ruff check, ruff format --check, and ty all clean (the one pre-existing
ty warning in dialog_evaluator.py is unrelated to this PR).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: jrob5756 <jasont.robert@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(providers): experimental provider tier + claude-agent-sdk follow-ups (#241) (#242)
* feat(providers): experimental provider tier + claude-agent-sdk follow-ups (#241)
Two related threads landed together.
## Part 1 — Experimental provider tier (#241)
Adds a `ProviderCapabilities` Pydantic descriptor (`src/conductor/providers/
capabilities.py`) that every `AgentProvider` declares as a class-level
`CAPABILITIES` attribute. `conductor validate` now cross-checks workflow
features against each agent's resolved provider, surfacing silent capability
mismatches at validate time rather than at runtime.
Cross-check matrix (issue #241):
| Mismatch | Severity |
|---|---|
| `runtime.mcp_servers` + provider `mcp_tools=False` (and used by an LLM agent) | error |
| Non-empty agent `tools:` + `workflow_tools_passthrough=False` | error |
| `reasoning.effort` (or workflow default) not in `reasoning_effort` tuple | error |
| `agent.output` + `structured_output="none"` | error |
| `agent.output` + experimental + `prompt_injection` (stable providers silent) | warning |
| Concurrent provider in parallel group OR for_each with `max_concurrent > 1` | error |
| Agent `max_session_seconds` + `max_session_seconds=False` | error |
Runtime surfacing:
- `workflow_started` event gains a `providers` block (per-provider tier,
upstream pin, maintainer, full capability dump) and each agent entry
gains `provider_name`.
- CLI prints one Rich-bordered banner per unique experimental provider per
run, with auto-generated limitations from the descriptor. Idempotent
across resume replays via a `run_id`-scoped guard set.
- Web dashboard renders a yellow "exp" badge on agent nodes whose
resolved provider has `tier: experimental`.
Provider declarations:
- `copilot` — `tier: stable`, mcp_tools, tools passthrough, streaming,
reasoning effort, prompt_injection structured output, interrupt,
max_session_seconds, checkpoint_resume, usage tracking.
- `claude` — `tier: stable`, native structured output, no streaming
(Phase 1), no checkpoint_resume (stateless API).
- `claude-agent-sdk` — `tier: experimental` with declared carve-outs:
no MCP, no tools allowlist, no reasoning effort wiring, prompt_injection
schema, no checkpoint_resume.
Docs:
- `docs/providers/experimental.md` — full stability policy with promotion
criteria (full parity caps, named maintainer, ≥6mo green real-API CI,
upstream ≥1.0 or stable 0.x, example workflow).
- AGENTS.md gains an "Experimental Providers" section listing the allowed
carve-outs and the non-negotiable rules.
- `docs/providers/comparison.md` adds a Tier row.
- `examples/experimental-claude-agent-sdk.yaml` exercises the provider
end-to-end (validates against `make validate-examples`).
## Part 2 — `claude-agent-sdk` follow-up fixes (PR #104 review)
All 12 findings from the post-merge review on PR #104 addressed:
Critical:
1. `tools: []` no longer silently grants the `claude_code` preset — empty
list disables all tools, non-empty list refused loudly (workflow tool
names don't translate to Claude CLI tool IDs).
2. Factory raises `ProviderError` on workflow-level `mcp_servers`,
`temperature`, or `max_tokens` instead of silently dropping them.
3. `agent_turn_start` events now fire in the correct order:
`awaiting_model` immediately before each model call, `{turn: N}` at
the start of each iteration (was: after response, at iteration end).
4. Token accounting no longer double-counts. `ResultMessage.usage` is
the cumulative session total per the SDK; per-AssistantMessage usage
is the partial-output fallback only.
5. Error suggestions classified by exception type (CLI not found, auth,
rate limit, network, parse, generic CLI) instead of always advising
"check the claude CLI is installed".
6. Doc claim "SDK handles retries" corrected — the SDK retries only
filesystem ops, never API failures. Workflow-level `retry:` is now
the authoritative retry mechanism (driven by classified `is_retryable`).
Important:
7. `max_session_seconds` now enforced (was dead code) via `time.monotonic`
between SDK messages. Per-agent override honored via the standard
`agent.max_session_seconds ?? provider.default` pattern.
8. `asyncio.CancelledError` re-raised before the bare `except Exception`
so interrupts propagate.
9. `is_retryable` classified by `ResultMessage.stop_reason` and
`api_error_status` (429/5xx/overload/network → retryable; auth/400 →
not) instead of hardcoded `False`.
10. `validate_connection()` now probes the bundled CLI, `shutil.which`,
and the SDK's hardcoded fallback locations (was a no-op).
11. Parse failure when `agent.output` is declared raises `ValidationError`
instead of silently wrapping non-JSON as `{"response": ...}`. Partial
output (interrupt) still tolerates the wrapper.
12. `comparison.md` regression verified: pre-#104 already said "No (Phase 1)"
for Claude streaming, so no revert needed.
Test coverage:
- Added regression tests for every fix above (85 → 158 tests in
`test_claude_agent_sdk.py`, plus new test files for capabilities,
validator cross-checks, banner, workflow_started provider tier).
- Polish: extracted magic numbers to named constants, added Google-style
docstrings to helpers, wrapped CLI imports in `try/except ImportError`
for library use, synced README pin, corrected default model from
`claude-sonnet-4-6` (non-existent) to `claude-sonnet-4-5`.
## Verification
- 3311 tests pass, 16 skipped (was 3256 pre-PR)
- ruff clean; ruff format clean
- ty `src` clean (2 pre-existing warnings unrelated to this PR)
- `make validate-examples` passes
- Frontend `tsc -b` + `vite build` clean
Closes #241.
* fix(providers): silence ty unresolved-import for inner claude_agent_sdk import
The validate_connection probe re-imports claude_agent_sdk to locate the
bundled CLI binary. CI's Type Check job runs without the optional
claude-agent-sdk extra installed (`uv sync --group dev`), so ty cannot
resolve the import and exits non-zero. Mirror the top-level import's
`# ty: ignore[unresolved-import]` directive on the inner import.
* review: address findings from pr-review sweep (#242)
Six review agents (code-reviewer, pr-test-analyzer, silent-failure-hunter,
type-design-analyzer, comment-analyzer, dead-code-finder) flagged the
following — all addressed in this commit:
## Critical bugs (real code defects)
- `capabilities.py:251-275`: deleted duplicated docstring + try/except left
by a botched earlier edit. Reported by every reviewer.
- `claude_agent_sdk.py:607`: deleted duplicate @staticmethod decorator on
_build_output.
## Silent failures
- `workflow.py:_record_provider`: log a warning on capability resolution
failure (was completely silent); catch ImportError too (was only catching
KeyError + AttributeError, so a broken provider module would crash the
engine despite the comment claiming graceful degradation).
- `cli/run.py:_maybe_print_experimental_banner`: narrow the `except
Exception: pass` to `except (ValidationError, TypeError)` with a logger
warning. Was swallowing every error class including real bugs.
- `claude_agent_sdk.py:_build_output`: the catch-all `else` for unexpected
`structured_output` types now raises ValidationError when the agent
declared an output schema (was silently coercing to `{"response": ...}`).
- `claude_agent_sdk.py:879`: removed the tautological `"5" in msg and …`
guard (any of 500/502/503/504 already contains "5").
## Parity gaps
- `engine/workflow.py:build_workflow_started_data`: walk `config.for_each`
so for_each inline agent providers appear in the `providers` block.
Without this, a workflow using `claude-agent-sdk` only inside a for_each
group got NO banner and NO dashboard exp badge.
- `cli/run.py`: banner now routes through `_verbose_console` (the silent-
aware Console subclass) instead of a fresh `Console(stderr=True)`, so
`--silent` suppresses it consistently with every other progress print.
- `config/validator.py`: workflow-level `runtime.max_session_seconds` is
now cross-checked against `caps.max_session_seconds` (was only checked
per-agent), with the same "providers actually in use" logic as the
mcp_servers check.
## Type hardening
- `base.py`: `__init_subclass__` enforces `CAPABILITIES: ProviderCapabilities`
at class-definition time on every non-abstract subclass. Test fakes opt
out with `abstract=True`. Turns the lazy resolver check into an import-
time error so missing/mistyped descriptors cannot ship.
- `capabilities.py`: Pydantic `field_validator` rejects empty
`reasoning_effort=()` (documented as invalid but previously structurally
legal — would silently pass every per-level membership check).
- `engine/workflow.py`: dropped the `tier="unknown"` wire-format drift.
Replaced with a `status: "ok" | "unresolved"` discriminator so the
`tier` field stays constrained to the same Literal as
`ProviderCapabilities.tier` end-to-end.
- `events.ts`: `ProviderMetadata` now uses `status` discriminator + nullable
`tier`, matching the Python schema. Dropped the opaque
`capabilities: Record<string, unknown>` field that no frontend consumed.
- `workflow-store.ts`: dropped duplicate `ProviderMetadata` declaration;
re-exported from `types/events.ts` (single source of truth).
- `cli/run.py`: banner re-resolves capabilities from the provider name
instead of round-tripping the dump through the wire — keeps the JSONL
payload lean and the limitation logic in one place.
## Prose cleanup
- Removed stale "(Phase 1)" attribution in claude.py CAPABILITIES.
- Removed line-number reference to upstream `types.py:1676` (brittle).
- Removed "The previous implementation returned …" history note in
_classify_error_suggestion docstring.
- Updated `_validate_provider_capabilities` docstring to reference
`docs/providers/experimental.md` (the durable doc) instead of "the plan"
and "rubber-duck design review" (ephemeral).
- Corrected `_build_output_format` docstring misattribution (claimed
`_build_output` "tolerates missing keys via response wrapping" — actually
schema validation is just not enforced).
- Acknowledged the `_log_event_verbose` defensive-import redundancy.
- Documented in `_build_unimplemented_placeholder` that the experimental
banner WILL fire for not-yet-implemented providers (intentional).
- Removed redundant `__all__` convenience comment.
- Made CancelledError re-raise comment specific to the interrupt-handler
contract instead of generic Python advice.
- Moved load-bearing tool-result-truncation comment directly onto its
constant; clarified the verbose-only previews are display-only.
## Test coverage added
- Per-agent provider override scenarios in parallel groups + for_each
(both safe-override and unsafe-override directions).
- Workflow-level `runtime.max_session_seconds` validation (3 cases).
- For_each inline experimental provider recorded in workflow_started.
- Banner suppression when `verbose_mode=False` (--silent contract).
- Banner with multiple unique experimental providers in one workflow.
- Multi-error validator aggregation (independent violations both reported).
- `__init_subclass__` enforcement (3 cases: missing, wrong type, abstract
opt-out).
- `reasoning_effort=()` rejection.
- `_classify_error_suggestion` CLIConnectionError + ProcessError generic
fallback branches.
- `_is_retryable_result` text-fallback branch (when stop_reason and
api_error_status are both None).
- `_build_error_message` direct unit tests (full payload + empty
fallback).
- Tool-result 500-char truncation regression test.
- `_safe_callback` swallowing — failing event_callback must not abort
execution.
## Test fakes updated
- `tests/test_integration/test_mixed_providers.py:MockProvider`
- `tests/test_providers/test_registry.py:MockProvider`
- `tests/test_engine/test_workflow_interrupt.py:TestMockProvider`
- `src/conductor/cli/run.py:_MockProvider` (used by dry-run)
all now declare `abstract=True` to opt out of the CAPABILITIES requirement.
## Verification
- 3336 tests pass, 16 skipped (was 3324 → +12 new tests)
- ruff clean; ruff format clean
- ty check src: 2 pre-existing warnings, exit 0
- Frontend tsc -b + vite build: clean
* Fix merging error
* Add pydantic_deep CAPABILITIES
* Add deepseek example
---------
Co-authored-by: Jason Robert <jasont.robert@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Lucio C Tinoco <lucio.tinoco@gmail.com>
Co-authored-by: Lucio Cunha Tinoco <lucioti@microsoft.com>
Co-authored-by: Lester Sanchez <94066823+lesandiz@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: David Gómez <ldavid.gomez@gmail.com>
tima-110
pushed a commit
to tima-110/conductor
that referenced
this pull request
Jun 11, 2026
…put (microsoft#237) * fix(human_gate): nest prompt_for values under additional_input in context output Collected values from prompt_for were spread flat into the gate output dict, making them inaccessible at the intended nested path and allowing silent corruption of the `selected` field when the prompt_for field name was "selected". * fix(context): support nested explicit input projection * Revert "fix(context): support nested explicit input projection" This reverts commit 3d95cba. * docs(human_gate): document additional_input nesting + add web/template tests Follow-up to the prompt_for nesting fix: * Update skill references (yaml-schema.md, authoring.md) to teach the new `<gate>.output.additional_input.<field>` access path instead of the old flat `<gate>.output.<field>` shape, and call out the `context: explicit` mode limitation (nested-shorthand input declarations are not supported — declare the parent and traverse in Jinja2). * Add a CHANGELOG `[Unreleased]` entry under Fixed (silent data-corruption bug) and Changed (BREAKING template shape change) with the migration one-liner and the explicit-mode caveat. * Add two new tests in TestWorkflowEngineHumanGates: - test_human_gate_web_response_nests_additional_input — stubs the web dashboard so the web task wins the race and confirms the second call site of `context.store` (via `_wait_for_web_gate`) also produces the nested shape, with no flat-spread leakage. - test_human_gate_additional_input_readable_via_template — runs a downstream agent whose prompt template reads `{{ ask_human.output.additional_input.answer }}` and asserts the rendered prompt seen by the mock handler contains the value, so the actual user-facing template contract (not just the internal context shape) is locked in. All 190 tests in test_gates + test_workflow + test_event_emission pass; ruff check, ruff format --check, and ty all clean (the one pre-existing ty warning in dialog_evaluator.py is unrelated to this PR). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: jrob5756 <jasont.robert@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jrob5756
added a commit
that referenced
this pull request
Jun 24, 2026
* feat(providers): add Hermes Agent provider via hermes-agent Python library
Adds HermesProvider backed by the hermes-agent package (NousResearch).
Like the Claude provider it is an optional dependency — a clear install
hint is raised if the package is missing.
- execute() wraps the sync AIAgent.run_conversation() in run_in_executor
so the provider stays fully async
- Always passes quiet_mode/skip_context_files/skip_memory for clean runs
- agent.model maps to AIAgent(model=), max_agent_iterations to max_iterations
- Structured output: appends JSON instruction to prompt when agent.output
is declared, parses result with parse_json_output + validate_output
- Races executor against interrupt_signal and max_session_seconds timeout
- Emits agent_turn_start / agent_message events for dashboard parity
- Token counts and model/provider surfaced from hermes result dict
Schema changes: "hermes" added to AgentDef.provider and
ProviderSettings.name literals; registry ProviderType updated to match.
21 new tests, all passing. No regressions in existing suite.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(providers): add auth_token and base_url support to Claude provider
Enables routing the Claude provider through a gateway (e.g. Databricks
AI Gateway, LiteLLM) using bearer-token auth instead of a raw API key.
- ClaudeProvider gains auth_token and base_url constructor params;
both are forwarded to AsyncAnthropic() only when explicitly set,
letting the SDK's own env-var fallbacks (ANTHROPIC_AUTH_TOKEN,
ANTHROPIC_BASE_URL) still work when params are omitted
- ProviderSettings gains an auth_token (SecretStr, claude-only) field;
base_url and api_key are now also accepted for name='claude'
- factory.py extracts auth_token/base_url from provider_settings and
forwards them to ClaudeProvider when provider_type == 'claude'
- Validator updated: copilot-only fields still reject on other providers;
auth_token rejects on non-claude; hermes/openai-agents still reject
base_url/api_key with a clear "not yet implemented" message
Example YAML:
runtime:
provider:
name: claude
base_url: https://my-databricks-host/serving-endpoints/...
auth_token: ${DATABRICKS_TOKEN}
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: external workflow friction - minimal evidence-anchored fixes (#232)
* fix(parser): use greedy fence regex for JSON with embedded backticks
parse_json_output (executor/output.py) and _extract_json
(providers/copilot.py) both used a non-greedy fenced-block regex:
r'`(?:json)?\s*\n?(.*?)\n?`'
When an agent emitted JSON whose string values contained triple-backtick
substrings (common for Markdown- or shell-snippet-bearing payloads), the
first inner ` closed the match prematurely. The extracted substring was
invalid JSON, triggering parse-recovery loops and token waste -- the
exact failure mode reported in the external-workflow-friction
brainstorm Issue #1.
Fix: switch both call sites to a greedy capture with re.DOTALL:
r'`(?:json)?\s*\n(.*)\n`'
With DOTALL the greedy '.*' terminates at the last triple-backtick in
the string, which is the correct boundary. The existing first-{/[
heuristic and brace-match fallback remain as final attempts; the
canonical json.loads call at the end is the unchanged failure point
for genuinely malformed JSON.
Test: tests/test_executor/test_output.py gains
test_parse_json_with_triple_backticks_inside_string which fails on main
and passes after this commit. The malformed-input test is retained as a
regression guard for the unchanged error message.
Plan: docs/projects/usability-features/external-workflow-friction.plan.md
(Item 1, FR-1/FR-2/FR-3).
* fix(cli): abort --web-bg before fork when workflow has human_gate
conductor run --web-bg and conductor resume --web-bg forked a detached
background process to run the workflow. When the workflow reached a
human_gate step, Prompt.ask() read from the now-closed stdin and the
child crashed with EOFError. The parent only saw 'Background process
exited immediately with code 1' -- nothing pointed at the actual
incompatibility, --skip-gates, or the foreground --web alternative.
Fix: add _abort_web_bg_if_human_gate() helper in cli/app.py that loads
the workflow, walks config.agents, and if any 'type: human_gate' is
present (and --skip-gates is not set) aborts with the four-option
guidance message before launch_background() forks anything. Call site
added to both 'run' and 'resume' per the run/resume parity rule in
AGENTS.md.
Tests: tests/test_cli/test_web_flags.py gains a TestWebBgHumanGateValidation
class with three CliRunner tests:
- test_web_bg_with_human_gate_aborts_before_fork
- test_web_bg_with_human_gate_and_skip_gates_proceeds
- test_resume_web_bg_with_human_gate_aborts_before_fork
All three mock launch_background, run in-process, and produce no
subprocess. Deterministic.
Plan: docs/projects/usability-features/external-workflow-friction.plan.md
(Item 4, FR-6).
* docs: omit-output guidance, --web-bg gate constraint, brainstorm + plan
Three documentation updates plus the planning artifacts behind this
fix cluster.
1. docs/workflow-syntax.md: new 'Choosing whether to declare output:'
subsection. Explains that declaring 'output:' both injects a JSON
schema instruction AND parses the response, and that omitting it
lets downstream agents read '<agent>.output.result' for prose or
large/nested JSON. Closes the documentation gap behind
external-workflow-friction Issue #2.
2. docs/cli-reference.md: --web-bg section now documents the
human_gate incompatibility and the four supported options matching
the new pre-fork validation in cli/app.py.
3. CHANGELOG.md [Unreleased]: entries for the fence regex fix, the
--web-bg human_gate validation, and the docs additions.
4. docs/projects/usability-features/: brainstorm and plan documents
that produced this fix cluster. The plan is the source of truth for
which brainstorm items were scoped in (G1/G3/G4 -- the three
commits in this branch) versus explicitly deferred (NG1-NG7 in
the plan, with thresholds to reopen). Kept in-tree so future
contributors can answer 'why was item X not done' without spelunking
PR history.
* fix(cli): emit --web-bg human_gate abort via plain echo + tolerate stderr in test
Click 8.3+ separates result.output (stdout) from result.stderr, and Typer renders BadParameter as a Rich panel that word-wraps narrow in CI runners, splitting '--skip-gates' across box-border lines so the literal substring is absent from the captured output. Switch the abort path to plain typer.echo(err=True) + typer.Exit so the message renders verbatim, and update the asserting tests to read result.stderr too.
* fix: address PR #232 review — multi-fence parse, for_each gate, resume gate
Addresses inline review feedback from jrob5756 on PR #232 (all flagged as
non-blocking but worth fixing):
1. executor/output.py + providers/copilot.py — multi-fence regression:
The greedy regex from the original PR fixed backticks-in-string but
broke responses with multiple fenced JSON blocks (the greedy capture
spans from the first '\\\' to the last, swallowing everything in
between and failing json.loads with 'Extra data').
New two-stage strategy: try non-greedy re.findall + per-candidate
try-parse first (first valid wins → multi-block-safe), fall back to
the greedy single capture (recovers backticks-in-string).
Adds regression tests in both test_output.py and test_copilot.py
that pin 'first valid block wins' on the multi-block input from the
reviewer's repro.
2. cli/app.py _abort_web_bg_if_human_gate — for_each coverage gap:
The check only walked config.agents. A workflow with human_gate
declared inline inside a for_each.agent passed the guard and still
silent-crashed under --web-bg. Now also walks config.for_each and
inspects each group's inline agent. Regression test added.
3. cli/app.py resume command — checkpoint-only resume gap:
When the user ran 'conductor resume --from <ckpt> --web-bg' without
a workflow argument, resolved_workflow was None and the gate guard
was skipped entirely. Now reads workflow_path from the checkpoint
JSON and runs the same check. Regression test added.
Tests: 959 passed, 3 skipped (was 954 before this commit + 5 new).
Lint: ruff check + format clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Lucio Cunha Tinoco <lucioti@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: release 0.1.18 (#233)
Bump version to 0.1.18 and finalize CHANGELOG.
Adds previously undocumented entries for:
- type: set step (#226, closes #221)
- silent-aware _verbose_console (#223, closes #209)
Plus existing Unreleased entries for type: wait (#224), type: terminate
(#219), structured runtime.provider (#225), and external-workflow-friction
fixes (#232).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(providers): add claude-agent-sdk provider (#104)
* feat(providers): add claude-agent-sdk provider
Add a new provider that uses the claude-agent-sdk package to delegate
the agentic loop, tool execution, and structured output extraction to
the Claude Code CLI. Unlike the raw claude provider, this provider does
not manage its own retry logic, MCP servers, or tool wiring — these are
handled by the SDK runtime.
- New provider: ClaudeAgentSdkProvider with execute(), validate_connection(), close()
- Message dispatch via type(message).__name__ matching real SDK class names
- Tool result pairing: tracks pending tool_use IDs to emit agent_tool_complete
with actual results from ToolResultBlock (not fake nulls)
- Structured output via ClaudeAgentOptions.output_format (json_schema)
- Event callback parity: agent_turn_start, agent_message, agent_reasoning,
agent_tool_start, agent_tool_complete
- 22 tests using real SDK types (AssistantMessage, TextBlock, etc.)
- Updated schema, factory, __init__, docs, and example workflow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(claude-agent-sdk): enhance logging with verbose and full mode options
* fix: post-merge CI fixes for type-check and install-scripts
- providers/registry.py: add 'claude-agent-sdk' to ProviderType Literal
so _get_provider_type_for_agent's return type covers the new provider
- providers/claude_agent_sdk.py: type the missing-extra fallbacks as
Any (ty was treating None-typed query/ClaudeAgentOptions as the only
shape) and cast SDK message objects to Any in dispatch branches where
hasattr/getattr narrowing isn't enough for ty
- tests/test_providers/test_claude_agent_sdk.py: pytest.importorskip
the claude_agent_sdk module so collection succeeds in the base
install (install-scripts CI job runs without the optional extra)
* fix(typecheck): ty:ignore unresolved-import for optional claude_agent_sdk
The Type Check CI job runs uv sync --group dev without optional extras,
so the upstream claude_agent_sdk package isn't installed. The try/except
ImportError pattern handles this at runtime, but ty's static resolver
still complains. Add a ty:ignore directive on the import line.
Verified locally with the package uninstalled: 0 errors from ty.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jrob5756 <jasont.robert@gmail.com>
* fix(human_gate): nest prompt_for collected values under additional_input (#237)
* fix(human_gate): nest prompt_for values under additional_input in context output
Collected values from prompt_for were spread flat into the gate output dict,
making them inaccessible at the intended nested path and allowing silent
corruption of the `selected` field when the prompt_for field name was
"selected".
* fix(context): support nested explicit input projection
* Revert "fix(context): support nested explicit input projection"
This reverts commit 3d95cba.
* docs(human_gate): document additional_input nesting + add web/template tests
Follow-up to the prompt_for nesting fix:
* Update skill references (yaml-schema.md, authoring.md) to teach the
new `<gate>.output.additional_input.<field>` access path instead of
the old flat `<gate>.output.<field>` shape, and call out the
`context: explicit` mode limitation (nested-shorthand input
declarations are not supported — declare the parent and traverse in
Jinja2).
* Add a CHANGELOG `[Unreleased]` entry under Fixed (silent
data-corruption bug) and Changed (BREAKING template shape change)
with the migration one-liner and the explicit-mode caveat.
* Add two new tests in TestWorkflowEngineHumanGates:
- test_human_gate_web_response_nests_additional_input — stubs the
web dashboard so the web task wins the race and confirms the
second call site of `context.store` (via `_wait_for_web_gate`)
also produces the nested shape, with no flat-spread leakage.
- test_human_gate_additional_input_readable_via_template — runs a
downstream agent whose prompt template reads
`{{ ask_human.output.additional_input.answer }}` and asserts the
rendered prompt seen by the mock handler contains the value, so
the actual user-facing template contract (not just the internal
context shape) is locked in.
All 190 tests in test_gates + test_workflow + test_event_emission pass;
ruff check, ruff format --check, and ty all clean (the one pre-existing
ty warning in dialog_evaluator.py is unrelated to this PR).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: jrob5756 <jasont.robert@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(providers): experimental provider tier + claude-agent-sdk follow-ups (#241) (#242)
* feat(providers): experimental provider tier + claude-agent-sdk follow-ups (#241)
Two related threads landed together.
## Part 1 — Experimental provider tier (#241)
Adds a `ProviderCapabilities` Pydantic descriptor (`src/conductor/providers/
capabilities.py`) that every `AgentProvider` declares as a class-level
`CAPABILITIES` attribute. `conductor validate` now cross-checks workflow
features against each agent's resolved provider, surfacing silent capability
mismatches at validate time rather than at runtime.
Cross-check matrix (issue #241):
| Mismatch | Severity |
|---|---|
| `runtime.mcp_servers` + provider `mcp_tools=False` (and used by an LLM agent) | error |
| Non-empty agent `tools:` + `workflow_tools_passthrough=False` | error |
| `reasoning.effort` (or workflow default) not in `reasoning_effort` tuple | error |
| `agent.output` + `structured_output="none"` | error |
| `agent.output` + experimental + `prompt_injection` (stable providers silent) | warning |
| Concurrent provider in parallel group OR for_each with `max_concurrent > 1` | error |
| Agent `max_session_seconds` + `max_session_seconds=False` | error |
Runtime surfacing:
- `workflow_started` event gains a `providers` block (per-provider tier,
upstream pin, maintainer, full capability dump) and each agent entry
gains `provider_name`.
- CLI prints one Rich-bordered banner per unique experimental provider per
run, with auto-generated limitations from the descriptor. Idempotent
across resume replays via a `run_id`-scoped guard set.
- Web dashboard renders a yellow "exp" badge on agent nodes whose
resolved provider has `tier: experimental`.
Provider declarations:
- `copilot` — `tier: stable`, mcp_tools, tools passthrough, streaming,
reasoning effort, prompt_injection structured output, interrupt,
max_session_seconds, checkpoint_resume, usage tracking.
- `claude` — `tier: stable`, native structured output, no streaming
(Phase 1), no checkpoint_resume (stateless API).
- `claude-agent-sdk` — `tier: experimental` with declared carve-outs:
no MCP, no tools allowlist, no reasoning effort wiring, prompt_injection
schema, no checkpoint_resume.
Docs:
- `docs/providers/experimental.md` — full stability policy with promotion
criteria (full parity caps, named maintainer, ≥6mo green real-API CI,
upstream ≥1.0 or stable 0.x, example workflow).
- AGENTS.md gains an "Experimental Providers" section listing the allowed
carve-outs and the non-negotiable rules.
- `docs/providers/comparison.md` adds a Tier row.
- `examples/experimental-claude-agent-sdk.yaml` exercises the provider
end-to-end (validates against `make validate-examples`).
## Part 2 — `claude-agent-sdk` follow-up fixes (PR #104 review)
All 12 findings from the post-merge review on PR #104 addressed:
Critical:
1. `tools: []` no longer silently grants the `claude_code` preset — empty
list disables all tools, non-empty list refused loudly (workflow tool
names don't translate to Claude CLI tool IDs).
2. Factory raises `ProviderError` on workflow-level `mcp_servers`,
`temperature`, or `max_tokens` instead of silently dropping them.
3. `agent_turn_start` events now fire in the correct order:
`awaiting_model` immediately before each model call, `{turn: N}` at
the start of each iteration (was: after response, at iteration end).
4. Token accounting no longer double-counts. `ResultMessage.usage` is
the cumulative session total per the SDK; per-AssistantMessage usage
is the partial-output fallback only.
5. Error suggestions classified by exception type (CLI not found, auth,
rate limit, network, parse, generic CLI) instead of always advising
"check the claude CLI is installed".
6. Doc claim "SDK handles retries" corrected — the SDK retries only
filesystem ops, never API failures. Workflow-level `retry:` is now
the authoritative retry mechanism (driven by classified `is_retryable`).
Important:
7. `max_session_seconds` now enforced (was dead code) via `time.monotonic`
between SDK messages. Per-agent override honored via the standard
`agent.max_session_seconds ?? provider.default` pattern.
8. `asyncio.CancelledError` re-raised before the bare `except Exception`
so interrupts propagate.
9. `is_retryable` classified by `ResultMessage.stop_reason` and
`api_error_status` (429/5xx/overload/network → retryable; auth/400 →
not) instead of hardcoded `False`.
10. `validate_connection()` now probes the bundled CLI, `shutil.which`,
and the SDK's hardcoded fallback locations (was a no-op).
11. Parse failure when `agent.output` is declared raises `ValidationError`
instead of silently wrapping non-JSON as `{"response": ...}`. Partial
output (interrupt) still tolerates the wrapper.
12. `comparison.md` regression verified: pre-#104 already said "No (Phase 1)"
for Claude streaming, so no revert needed.
Test coverage:
- Added regression tests for every fix above (85 → 158 tests in
`test_claude_agent_sdk.py`, plus new test files for capabilities,
validator cross-checks, banner, workflow_started provider tier).
- Polish: extracted magic numbers to named constants, added Google-style
docstrings to helpers, wrapped CLI imports in `try/except ImportError`
for library use, synced README pin, corrected default model from
`claude-sonnet-4-6` (non-existent) to `claude-sonnet-4-5`.
## Verification
- 3311 tests pass, 16 skipped (was 3256 pre-PR)
- ruff clean; ruff format clean
- ty `src` clean (2 pre-existing warnings unrelated to this PR)
- `make validate-examples` passes
- Frontend `tsc -b` + `vite build` clean
Closes #241.
* fix(providers): silence ty unresolved-import for inner claude_agent_sdk import
The validate_connection probe re-imports claude_agent_sdk to locate the
bundled CLI binary. CI's Type Check job runs without the optional
claude-agent-sdk extra installed (`uv sync --group dev`), so ty cannot
resolve the import and exits non-zero. Mirror the top-level import's
`# ty: ignore[unresolved-import]` directive on the inner import.
* review: address findings from pr-review sweep (#242)
Six review agents (code-reviewer, pr-test-analyzer, silent-failure-hunter,
type-design-analyzer, comment-analyzer, dead-code-finder) flagged the
following — all addressed in this commit:
## Critical bugs (real code defects)
- `capabilities.py:251-275`: deleted duplicated docstring + try/except left
by a botched earlier edit. Reported by every reviewer.
- `claude_agent_sdk.py:607`: deleted duplicate @staticmethod decorator on
_build_output.
## Silent failures
- `workflow.py:_record_provider`: log a warning on capability resolution
failure (was completely silent); catch ImportError too (was only catching
KeyError + AttributeError, so a broken provider module would crash the
engine despite the comment claiming graceful degradation).
- `cli/run.py:_maybe_print_experimental_banner`: narrow the `except
Exception: pass` to `except (ValidationError, TypeError)` with a logger
warning. Was swallowing every error class including real bugs.
- `claude_agent_sdk.py:_build_output`: the catch-all `else` for unexpected
`structured_output` types now raises ValidationError when the agent
declared an output schema (was silently coercing to `{"response": ...}`).
- `claude_agent_sdk.py:879`: removed the tautological `"5" in msg and …`
guard (any of 500/502/503/504 already contains "5").
## Parity gaps
- `engine/workflow.py:build_workflow_started_data`: walk `config.for_each`
so for_each inline agent providers appear in the `providers` block.
Without this, a workflow using `claude-agent-sdk` only inside a for_each
group got NO banner and NO dashboard exp badge.
- `cli/run.py`: banner now routes through `_verbose_console` (the silent-
aware Console subclass) instead of a fresh `Console(stderr=True)`, so
`--silent` suppresses it consistently with every other progress print.
- `config/validator.py`: workflow-level `runtime.max_session_seconds` is
now cross-checked against `caps.max_session_seconds` (was only checked
per-agent), with the same "providers actually in use" logic as the
mcp_servers check.
## Type hardening
- `base.py`: `__init_subclass__` enforces `CAPABILITIES: ProviderCapabilities`
at class-definition time on every non-abstract subclass. Test fakes opt
out with `abstract=True`. Turns the lazy resolver check into an import-
time error so missing/mistyped descriptors cannot ship.
- `capabilities.py`: Pydantic `field_validator` rejects empty
`reasoning_effort=()` (documented as invalid but previously structurally
legal — would silently pass every per-level membership check).
- `engine/workflow.py`: dropped the `tier="unknown"` wire-format drift.
Replaced with a `status: "ok" | "unresolved"` discriminator so the
`tier` field stays constrained to the same Literal as
`ProviderCapabilities.tier` end-to-end.
- `events.ts`: `ProviderMetadata` now uses `status` discriminator + nullable
`tier`, matching the Python schema. Dropped the opaque
`capabilities: Record<string, unknown>` field that no frontend consumed.
- `workflow-store.ts`: dropped duplicate `ProviderMetadata` declaration;
re-exported from `types/events.ts` (single source of truth).
- `cli/run.py`: banner re-resolves capabilities from the provider name
instead of round-tripping the dump through the wire — keeps the JSONL
payload lean and the limitation logic in one place.
## Prose cleanup
- Removed stale "(Phase 1)" attribution in claude.py CAPABILITIES.
- Removed line-number reference to upstream `types.py:1676` (brittle).
- Removed "The previous implementation returned …" history note in
_classify_error_suggestion docstring.
- Updated `_validate_provider_capabilities` docstring to reference
`docs/providers/experimental.md` (the durable doc) instead of "the plan"
and "rubber-duck design review" (ephemeral).
- Corrected `_build_output_format` docstring misattribution (claimed
`_build_output` "tolerates missing keys via response wrapping" — actually
schema validation is just not enforced).
- Acknowledged the `_log_event_verbose` defensive-import redundancy.
- Documented in `_build_unimplemented_placeholder` that the experimental
banner WILL fire for not-yet-implemented providers (intentional).
- Removed redundant `__all__` convenience comment.
- Made CancelledError re-raise comment specific to the interrupt-handler
contract instead of generic Python advice.
- Moved load-bearing tool-result-truncation comment directly onto its
constant; clarified the verbose-only previews are display-only.
## Test coverage added
- Per-agent provider override scenarios in parallel groups + for_each
(both safe-override and unsafe-override directions).
- Workflow-level `runtime.max_session_seconds` validation (3 cases).
- For_each inline experimental provider recorded in workflow_started.
- Banner suppression when `verbose_mode=False` (--silent contract).
- Banner with multiple unique experimental providers in one workflow.
- Multi-error validator aggregation (independent violations both reported).
- `__init_subclass__` enforcement (3 cases: missing, wrong type, abstract
opt-out).
- `reasoning_effort=()` rejection.
- `_classify_error_suggestion` CLIConnectionError + ProcessError generic
fallback branches.
- `_is_retryable_result` text-fallback branch (when stop_reason and
api_error_status are both None).
- `_build_error_message` direct unit tests (full payload + empty
fallback).
- Tool-result 500-char truncation regression test.
- `_safe_callback` swallowing — failing event_callback must not abort
execution.
## Test fakes updated
- `tests/test_integration/test_mixed_providers.py:MockProvider`
- `tests/test_providers/test_registry.py:MockProvider`
- `tests/test_engine/test_workflow_interrupt.py:TestMockProvider`
- `src/conductor/cli/run.py:_MockProvider` (used by dry-run)
all now declare `abstract=True` to opt out of the CAPABILITIES requirement.
## Verification
- 3336 tests pass, 16 skipped (was 3324 → +12 new tests)
- ruff clean; ruff format clean
- ty check src: 2 pre-existing warnings, exit 0
- Frontend tsc -b + vite build: clean
* chore(deps): Bump starlette from 0.52.1 to 1.0.1 (#246)
Bumps [starlette](https://github.com/Kludex/starlette) from 0.52.1 to 1.0.1.
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](Kludex/starlette@0.52.1...1.0.1)
---
updated-dependencies:
- dependency-name: starlette
dependency-version: 1.0.1
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(context): support nested explicit input projection (#239)
* fix(context): support nested explicit input projection
* fix(context): align nested explicit input validation with runtime
* feat(providers): add Hermes Agent provider via hermes-agent Python library
Adds HermesProvider backed by the hermes-agent package (NousResearch).
Like the Claude provider it is an optional dependency — a clear install
hint is raised if the package is missing.
- execute() wraps the sync AIAgent.run_conversation() in run_in_executor
so the provider stays fully async
- Always passes quiet_mode/skip_context_files/skip_memory for clean runs
- agent.model maps to AIAgent(model=), max_agent_iterations to max_iterations
- Structured output: appends JSON instruction to prompt when agent.output
is declared, parses result with parse_json_output + validate_output
- Races executor against interrupt_signal and max_session_seconds timeout
- Emits agent_turn_start / agent_message events for dashboard parity
- Token counts and model/provider surfaced from hermes result dict
Schema changes: "hermes" added to AgentDef.provider and
ProviderSettings.name literals; registry ProviderType updated to match.
21 new tests, all passing. No regressions in existing suite.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(providers): add Hermes provider docs, example, and skills references
- New docs/providers/hermes.md covering quick start, model format,
runtime config, structured output, tool use, limitations, and
troubleshooting
- New examples/hermes-simple.yaml validated Q&A workflow
- Updated docs/providers/comparison.md to include Hermes column
- Updated all skills references (authoring, yaml-schema, execution,
SKILL.md) to list hermes alongside copilot/claude
- Fixed test_backward_compatibility to exclude hermes-named example files
from the copilot-only provider assertion
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(providers): hermes smoke-test fixes — system_prompt, toolsets, max_tokens, base_url/api_key routing
- Forward `agent.system_prompt` as `system_message=` kwarg to `run_conversation()`
- Map `agent.tools` to AIAgent `enabled_toolsets`: None→omit (all tools), []→empty allowlist (0 tools), [list]→named toolsets
- Forward `max_tokens` and `temperature` from provider init to AIAgent kwargs (were previously ignored)
- Forward `base_url` and `api_key` from provider init to AIAgent for custom endpoint routing
- Include resolved model name in ProviderError on failed result for easier debugging
- Relax `ProviderSettings._check_field_compatibility` to allow `base_url`/`api_key` for `name="hermes"` (was copilot-only)
- Update factory `case "hermes"` to pass `max_tokens`, `temperature`, `max_session_seconds`, and provider_settings credentials
- Add 12 new tests (system_prompt, tools mapping, forwarded params, provider settings)
- Update hermes.md: Runtime Configuration table, Toolset Control section, Model Routing section
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(skills): update yaml-schema reference — hermes allows base_url/api_key in ProviderSettings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(providers): sync hermes docs and skills with smoke-test fixes
- hermes.md: update model-not-found error format to show model name in message
- yaml-schema.md: note that tools: maps to enabled_toolsets (hermes toolset names) for hermes provider
- yaml-schema.md: clarify base_url/api_key are copilot+hermes; bearer_token is copilot-only
- execution.md: remove false claim that Claude drops system_prompt (it doesn't; hermes did, now fixed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(readme): add Hermes as a first-class provider
- Features bullet: mention Hermes alongside Copilot and Claude
- Providers table: add Hermes column with pricing, context, tools, streaming, best-for
- Add "Using Hermes" snippet with install hint
- Docs table: add Hermes provider doc link; update comparison description
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: add Hermes to provider enumerations across configuration, CLI, and syntax docs
- configuration.md: add Hermes Provider section; update basic structure comment
- cli-reference.md: add hermes to --provider flag description
- workflow-syntax.md: add hermes to provider comment in schema example
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(providers): declare Hermes as experimental provider with ProviderCapabilities
- Add CAPABILITIES declaration to HermesProvider (tier=experimental,
no MCP, no tools passthrough, no streaming, prompt-injection schema)
- Register hermes in capabilities.py resolver (_PROVIDER_CLASS_PATHS)
- Add experimental banners to docs/providers/hermes.md, configuration.md
- Update skills references to include claude-agent-sdk + hermes with
experimental marker
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(providers): address PR review blockers for Hermes provider
- Remove agent.tools → enabled_toolsets passthrough (security regression:
Conductor tool names and hermes toolset names are different vocabularies)
- Wrap call_task.result() to catch raw third-party exceptions and raise
ProviderError with actionable suggestion
- Fix interrupt/timeout: add is_retryable classification, warn that
executor thread cannot be stopped, update error messages
- Raise _fire log level from DEBUG to WARNING so broken event sinks
are visible at default log levels
- Add factory branch test coverage for hermes (create, config forwarding,
SDK-unavailable rejection)
- Update test assertions to match new wrapped-exception behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(providers): address PR review should-fix items for Hermes provider
- Store default_reasoning_effort on self (was accepted but discarded)
- Replace deprecated asyncio.get_event_loop() with get_running_loop()
- Fix validate_connection() docstring to accurately describe behavior
- Simplify dead "copilot" check in ProviderSettings validator (already
excluded by outer if-block)
- Fix token-count fallback: use explicit None-check instead of `or` so
legitimate zero values are not collapsed
- Tighten import: catch ModuleNotFoundError for run_agent specifically,
let real dependency failures inside hermes-agent propagate
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): wire hermes streaming + reasoning callbacks, enable usage_tracking
Close 3 capability gaps using hermes-agent's native callback API:
- streaming_events: True — wire stream_delta_callback on AIAgent to emit
agent_message events incrementally as text deltas arrive
- agent_reasoning_events: True — wire reasoning_callback on AIAgent to
emit agent_reasoning events with thinking/CoT content
- usage_tracking: True — hermes result dict reliably includes
input_tokens/output_tokens/total_tokens on every call
The callbacks fire from the executor thread via the existing _fire helper
(thread-safe, swallows exceptions at WARNING level).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): add checkpoint_resume support to Hermes via conversation history
Hermes run_conversation() accepts a conversation_history parameter that
restores prior context. Leverage this for checkpoint resume:
- After each successful execute(), persist the messages list to a JSON
file in $TMPDIR/conductor/hermes-sessions/{agent_name}.json
- Expose get_session_ids() / set_resume_session_ids() matching the
Copilot provider contract — session IDs are file paths
- On resume, load the conversation history from disk and pass it to
run_conversation(); hermes handles context compression internally
- Add cleanup_sessions() for explicit teardown
- Update CAPABILITIES: checkpoint_resume=True
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): wire reasoning_effort through to hermes via reasoning_config
AIAgent accepts reasoning_config={"effort": "low|medium|high|xhigh"}
which hermes translates to the native API for each provider:
- Anthropic: extended thinking with budget_tokens / adaptive thinking
- GitHub Models/OpenAI: reasoning_effort in extra_body
- LM Studio: its own effort resolution
Wire resolve_reasoning_effort() (per-agent override → workflow default)
into the AIAgent constructor kwargs. Update CAPABILITIES to declare
support for all four effort levels.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): enable cooperative interrupt for Hermes via agent.interrupt()
AIAgent.interrupt() is a thread-safe method designed to be called from
another thread. It sets _interrupt_requested which the conversation loop
checks between each API call and tool iteration — a cooperative interrupt.
Wire this into our async interrupt handling:
- Keep a reference to the AIAgent instance created inside _run_sync
- When interrupt_signal fires (or timeout expires), call
hermes_agent.interrupt() which signals the loop to stop
- Hermes returns partial=True with whatever progress was made
Update CAPABILITIES: interrupt=True (now cooperative, not best-effort).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): add hermes_home profile support to ProviderSettings
Hermes loads its config (soul, memory, toolsets) from a home directory.
Add hermes_home field to ProviderSettings so workflows can specify which
profile to use:
runtime:
provider:
name: hermes
hermes_home: ~/.hermes-research
Implementation:
- ProviderSettings.hermes_home: str | None (hermes-only, validated)
- Factory extracts hermes_home and passes to HermesProvider
- HermesProvider uses hermes_constants.set_hermes_home_override() inside
_run_sync — this is a ContextVar, thread-safe for concurrent agents
- Override is reset in a finally block after each execution
Per-agent profile switching (different hermes_home per agent within a
single workflow) can be a follow-up via provider_options.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): align Hermes structured output with Copilot pattern
Replace the simple _JSON_INSTRUCTION append with the full Copilot-style
structured output handling:
- Build a complete prompt schema description from OutputField definitions
(same format as CopilotProvider._build_prompt_schema)
- Append schema + "MUST respond with JSON" instruction to the prompt
- On parse failure, run a recovery loop (up to 3 attempts) that sends a
follow-up message with the parse error, truncated response, and expected
schema — using conversation_history to maintain context
- Raise ProviderError with actionable suggestion after exhausting retries
This mirrors Copilot's _build_parse_recovery_prompt + retry pattern so
both prompt-injection providers behave consistently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(providers): set workflow_tools_passthrough=False and add hermes_toolsets
Conductor's per-agent tools: field contains workflow tool names that do not
translate to Hermes toolset names. Passing them through silently grants the
wrong tools — a security regression. Follow the claude_agent_sdk pattern:
reject non-empty tools lists at execute time, and provide hermes_toolsets
in ProviderSettings for authors who need to restrict available toolsets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(providers): route hermes temperature through request_overrides
AIAgent doesn't accept temperature as a direct constructor kwarg — passing
it directly would raise TypeError. Route through request_overrides which
is applied at the transport layer, matching how hermes handles temperature
internally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(skills): sync conductor skill with Hermes provider and nested input projection
Update skill references to reflect recent changes: Hermes now supports
temperature, max_tokens, reasoning_effort, and rejects per-agent tools:
(use hermes_toolsets in ProviderSettings instead). Add nested explicit
input projection syntax, hermes_home/hermes_toolsets to schema, and
openai-agents to the runtime provider list. Expand factory tests for
full Hermes config forwarding.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(examples): add Hermes provider functional test workflows
Two workflows exercising Hermes features end-to-end:
- hermes-features.yaml: structured output, reasoning effort, temperature,
max_tokens, tools:[], parallel group, nested input projection, set step
combo, conditional routing
- hermes-toolsets.yaml: hermes_toolsets provider restriction, for_each
dynamic fan-out, max_agent_iterations, max_session_seconds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(providers): stop forcing skip_memory/skip_context_files in Hermes provider
The provider hardcoded skip_memory=True and skip_context_files=True,
which stripped the agent's persona identity when using a Hermes profile
(hermes_home). These flags are now omitted by default (letting the
hermes-agent library defaults apply) and configurable via
hermes_skip_memory / hermes_skip_context_files in ProviderSettings.
Also fixes pre-existing lint issues on this branch (SIM102, SIM105,
E501, F401).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(providers): expand tilde in hermes_home before passing to SDK
The hermes-agent SDK does not expand ~ internally, causing persona
loading to fail when hermes_home uses a tilde path like
~/.hermes/profiles/chloe. Now uses Path.expanduser() before calling
set_hermes_home_override().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(providers): address remaining PR review items for Hermes provider
- Set interrupt=False in CAPABILITIES (raises ProviderError on interrupt
rather than returning partial output, matching the carve-out behavior)
- Make _parse_with_recovery async and route recovery calls through
run_in_executor to avoid blocking the event loop during parallel execution
- Apply hermes_home ContextVar override in recovery path (matching primary)
- Wrap recovery run_conversation exceptions in ProviderError so library
errors don't escape unwrapped
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Lucio C Tinoco <lucio.tinoco@gmail.com>
Co-authored-by: Lucio Cunha Tinoco <lucioti@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Jason Robert <jasont.robert@gmail.com>
Co-authored-by: Lester Sanchez <94066823+lesandiz@users.noreply.github.com>
Co-authored-by: David Gómez <ldavid.gomez@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: David Gómez <david.gomez@plainconcepts.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
When a
human_gateoption usesprompt_forto collect free-text input, the collected value is stored inGateResult.additional_input, but the workflow context output previously flattened that value into the gate output root.This meant downstream templates could not access the value through the intended nested form:
{{ gate.output.additional_input.field }}Previously, the value was exposed through the flattened path:
{{ gate.output.field }}That flat shape is unsafe because it can collide with reserved gate output keys. For example,
prompt_for: selectedcould overwrite the selected option value.Root cause
engine/workflow.pystored the gate result using a dict spread:This placed collected values at
gate.output.<field>rather thangate.output.additional_input.<field>.It also introduced a silent data-corruption risk: if
prompt_forused the field nameselected, the spread could overwrite the chosen option value with the user's raw text.The
gate_resolvedevent and_wait_for_web_gate()already useadditional_inputas a nested key, so the flat spread in workflow context storage was inconsistent with the rest of the implementation.Fix
Replace the flat spread with an explicit nested key:
Gates without
prompt_forcontinue to produce:{"additional_input": {}}Compatibility note
This changes the previously observable flat output shape for
prompt_forvalues, for example:{{ gate.output.answer }}That flat shape appears to be undocumented and unsafe because it can collide with reserved output keys such as
selected.This may affect workflows that were relying on the undocumented flattened form
gate.output.<prompt_for_field>.The canonical downstream reference after this fix is:
{{ gate.output.additional_input.answer }}Tests added
test_human_gate_stores_additional_input_nested_in_contextoutput.additional_input.answeris populated afterprompt_forinputtest_human_gate_no_prompt_for_has_empty_additional_inputprompt_forstill produceadditional_input: {}test_human_gate_prompt_for_named_selected_does_not_corrupt_selectedprompt_for: selecteddoes not overwrite the chosen option valueTests run
Total relevant validation:
Smoke result
CLI smoke confirmed:
Confirmed template access:
{{ ask_human.output.additional_input.answer }}renders:
Known limitation
In
context: explicit, the current input projection logic does not support nested shorthand declarations such as:The working explicit-mode declaration is to include the parent key:
and then let the template traverse into it:
{{ ask_human.output.additional_input.answer }}This is a generic context input-projection limitation, not specific to
human_gateorprompt_for, and is intentionally left out of this PR to keep the change focused.