Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,12 @@ make validate-examples # validate all examples
- `agent.py` - `AgentExecutor` handles prompt rendering, tool resolution, and output validation for single agents
- `script.py` - `ScriptExecutor` runs shell commands as workflow steps, capturing stdout/stderr/exit_code
- `set_step.py` - `SetExecutor` evaluates Jinja2 expressions for `type: set` steps and binds typed values into the workflow context (no LLM, no subprocess). Supports single `value:` and multi `values:` forms with auto / explicit `output_type:` coercion.
- `wait.py` - `WaitExecutor` pauses workflow execution for a parsed duration via `asyncio.sleep`. Races the sleep against the engine's `interrupt_event` so Esc/Ctrl+G cancels in-flight waits immediately; the workflow-level `limits.timeout_seconds` also cancels it via `LimitEnforcer.wait_for_with_timeout`. Output contract is strictly `{"waited_seconds": float}` per issue #218.
- `template.py` - Jinja2 template rendering
- `output.py` - JSON output parsing and schema validation

- **duration.py**: `parse_duration(value)` shared helper. Accepts plain `int`/`float` seconds, or strings with `ms`/`s`/`m`/`h` suffix. Raises `ValueError` (nests cleanly inside Pydantic `ValidationError`). Rejects booleans. Bounds enforcement (e.g. > 0, 24h cap) lives in callers so the parser can be reused.

- **providers/**: SDK provider abstraction
- `base.py` - `AgentProvider` ABC defining `execute()`, `validate_connection()`, `close()`
- `copilot.py` - GitHub Copilot SDK implementation
Expand All @@ -114,13 +117,14 @@ make validate-examples # validate all examples

1. CLI parses YAML via `config/loader.py` → `WorkflowConfig`
2. `WorkflowEngine` initializes with config and provider
3. Engine loops: find agent/parallel/for-each/script/set → execute → evaluate routes → next
3. Engine loops: find agent/parallel/for-each/script/set/wait → execute → evaluate routes → next
4. Parallel groups execute agents concurrently with context isolation (deep copy snapshot)
5. For-each groups resolve source arrays at runtime, inject loop variables (`{{ item }}`, `{{ _index }}`, `{{ _key }}`)
6. Script steps run shell commands via asyncio subprocess, expose stdout/stderr/exit_code to context
7. Set steps render Jinja2 expressions and bind typed values to context (no LLM, no subprocess) via the shared `WorkflowEngine._run_set_step` helper, which enforces `output:` schema in all three positions (main loop, parallel group, for-each iteration) and emits `set_started` / `set_completed` / `set_failed`
8. Routes evaluated via `Router` using Jinja2 or simpleeval expressions
9. Final output built from templates in `output:` section
8. Wait steps pause via `asyncio.sleep` (cancellable by interrupt or workflow timeout); expose `{"waited_seconds": float}` to context
9. Routes evaluated via `Router` using Jinja2 or simpleeval expressions
10. Final output built from templates in `output:` section

### Key Patterns

Expand Down
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased](https://github.com/microsoft/conductor/compare/v0.1.17...HEAD)

### Added
- New `type: wait` workflow step that pauses execution for a parsed
duration via in-process `asyncio.sleep`. Cross-platform — no shell
`sleep` dependency. Use for rate-limit cooldowns, polling intervals,
external-system catch-up, and demos. The `duration:` field accepts
plain numbers (seconds), suffixed strings (`"500ms"`, `"60s"`,
`"2.5m"`, `"1h"`), or a Jinja2 template that renders to one of
those (e.g. `"{{ workflow.input.poll_interval }}s"`). Schema enforces
`0 < duration <= 24h` and rejects boolean values pre-coercion.
`Esc` / `Ctrl+G` cancels in-progress waits immediately (the engine
races the sleep against the interrupt event), and the workflow-level
`limits.timeout_seconds` also cancels them. Wait steps emit
`wait_started` / `wait_completed` / `wait_failed` events alongside
the generic `agent_started` (with `agent_type: "wait"`), so existing
dashboards keyed on agent lifecycle pick them up automatically. The
dashboard adds a dedicated `WaitNode` (clock icon) and `WaitDetail`
panel that show the requested duration, actual elapsed time, reason,
and an "interrupted" indicator. The public output contract is strict
— only `{"waited_seconds": float}` is exposed to workflow context;
extra metadata lives in event payloads. Wait steps count toward
`limits.max_iterations` (each pause is one step) but are not subject
to `max_agent_iterations` (per-LLM-agent tool counter). Wait cannot
be used inside `parallel` or `for_each` groups. New `examples/wait-step.yaml`
demonstrates a polling pattern with a templated poll interval and
route loop-back
([#224](https://github.com/microsoft/conductor/pull/224),
closes [#218](https://github.com/microsoft/conductor/issues/218)).

## [0.1.17](https://github.com/microsoft/conductor/compare/v0.1.16...v0.1.17) - 2026-05-21

### Added
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,8 @@ See the [`examples/`](./examples/) directory for complete workflows:
| [design-review.yaml](./examples/design-review.yaml) | Human gate with loop pattern |
| [script-step.yaml](./examples/script-step.yaml) | Script step with exit_code routing |
| [set-step.yaml](./examples/set-step.yaml) | Set step deriving named values + boolean-routed branching |
| [wait-step.yaml](./examples/wait-step.yaml) | Wait step + script for a polling loop-back pattern |
| [wait-smoke.yaml](./examples/wait-smoke.yaml) | Minimal wait-only smoke test (no provider required) |

**More examples and running instructions:** [examples/README.md](./examples/README.md)

Expand Down
4 changes: 2 additions & 2 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,8 @@ agents:

Per-agent overrides always win over the workflow-wide default. The
`reasoning.effort` field is **only** valid on standard `agent`-type agents; it
is rejected on `script`, `human_gate`, and `workflow` agents (which do not call
a model).
is rejected on `script`, `human_gate`, `workflow`, and `wait` agents (which do
not call a model).

### Per-provider translation

Expand Down
69 changes: 67 additions & 2 deletions docs/workflow-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Agents are defined in the `agents` list. Each agent represents a unit of work.
agents:
- name: string # Required: Unique agent identifier
description: string # Optional: Purpose description
type: agent # agent | human_gate | script | workflow (default: agent)
type: agent # agent | human_gate | script | workflow | wait (default: agent)
model: string # Optional: Model identifier (e.g., 'claude-sonnet-4.5')

prompt: | # Required for type=agent: Agent instructions
Expand Down Expand Up @@ -281,6 +281,70 @@ routes:

**Environment variable note** — values in `env` are passed as-is to the subprocess (they are not rendered as Jinja2 templates). Use `${VAR}` syntax in the workflow YAML loader if you need environment variable substitution in env values.

### Wait Steps

Wait steps pause workflow execution for a parsed duration via in-process `asyncio.sleep`. Use them for rate-limit cooldowns, polling intervals, and external-system catch-up — cross-platform, no shell `sleep` dependency.

```yaml
agents:
- name: cooldown
type: wait
description: "Cool down between API bursts" # Optional
duration: 60s # Required: see "Duration format" below
reason: "Avoiding rate limit" # Optional: shown in dashboard
routes:
- to: next_step
```

**Duration format** — `duration` accepts:

- A plain `int` or `float` (seconds): `duration: 60`, `duration: 1.5`.
- A string with a unit suffix: `ms` (milliseconds), `s` (seconds), `m` (minutes), `h` (hours). Examples: `"500ms"`, `"60s"`, `"2.5m"`, `"1h"`.
- A Jinja2 template that renders to one of the above. Templated durations defer literal validation to runtime:

```yaml
duration: "{{ workflow.input.poll_interval_seconds }}s"
```

The resolved duration must be **greater than 0 and no more than 24 hours** (`86400s`). Longer pauses should reconsider `workflow.limits.timeout_seconds` first.

**Output structure** — wait step output is strict — only `waited_seconds` is exposed:

| Field | Type | Description |
|-------|------|-------------|
| `waited_seconds` | `number` | Wall-clock seconds actually slept (may be less than requested on interrupt) |

Access in templates: `{{ cooldown.output.waited_seconds }}`.

**Polling pattern** — wait composes with routing loop-backs to build polling workflows without writing any Python:

```yaml
agents:
- name: check_status
type: script
command: ./poll-status.sh
routes:
- to: process_result
when: "status == 'ready'"
- to: wait_then_retry

- name: wait_then_retry
type: wait
duration: "{{ workflow.input.poll_interval_seconds }}s"
routes:
- to: check_status # loop back

- name: process_result
# ...
```

**Cancellation** — `Esc` / `Ctrl+G` cancels an in-progress wait immediately (the engine races the sleep against the interrupt event). The workflow-level `limits.timeout_seconds` also cancels in-flight waits via the standard timeout path.

**Iteration counting** — wait steps count toward `workflow.limits.max_iterations` (each pause is one step). They are not subject to `max_agent_iterations`, which counts per-LLM-agent tool iterations.

**Restrictions** — wait steps cannot have `prompt`, `model`, `provider`, `tools`, `system_prompt`, `options`, `command`, `args`, `env`, `working_dir`, `timeout`, `workflow`, `input_mapping`, `max_depth`, `max_session_seconds`, `max_agent_iterations`, `retry`, `dialog`, `reasoning`, `timeout_seconds`, or `output`. Wait steps also cannot be used inside `parallel` groups or `for_each` groups.

See [`examples/wait-step.yaml`](../examples/wait-step.yaml) for a complete polling workflow.
### Set Steps

Set steps evaluate one or more Jinja2 expressions and bind the typed results into the workflow context. No LLM call, no subprocess, no I/O — they're pure context transformations. Use them to combine inputs, derive flags from prior outputs, compute defaults, or normalise a value once for many downstream prompts to share.
Expand Down Expand Up @@ -488,7 +552,7 @@ After the conversation, the agent re-executes with the dialog transcript as addi
| `dialog.trigger_prompt` | string | Yes | Criteria for the LLM evaluator to decide when dialog is needed |

**Behavior notes:**
- Dialog is supported on regular `agent` type only (not `human_gate`, `script`, or `workflow`)
- Dialog is supported on regular `agent` type only (not `human_gate`, `script`, `workflow`, or `wait`)
- In web dashboard mode, the dialog temporarily replaces the graph area with a chat interface
- When `--skip-gates` is set (e.g., CI/automation), dialogs are automatically skipped
- The evaluator prompt should describe *when* to trigger dialog, not *what* to ask — the evaluator generates the opening question from the agent's output context
Expand Down Expand Up @@ -782,6 +846,7 @@ workflow:
- Each agent execution counts as 1 iteration
- Parallel agents count individually (3 parallel agents = 3 iterations)
- Loop-back patterns increment the counter on each iteration
- Script steps and wait steps each count as 1 iteration

### Timeout Behavior

Expand Down
45 changes: 45 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,51 @@ Research workflow demonstrating multi-provider patterns. Demonstrates:
conductor run examples/multi-provider-research.yaml --input topic="Cloud computing"
```

## Step Types

### script-step.yaml

Script step with shell command, JSON output parsing, and `exit_code`-based routing.
Demonstrates:
- `type: script` agents (cross-platform shell command execution)
- Capturing stdout/stderr/exit_code
- Routing on `exit_code` (`when: "exit_code == 0"`)
- Passing script output to downstream LLM agents

```bash
conductor run examples/script-step.yaml
```

### wait-step.yaml

Polling pattern with a wait step and a routing loop-back. Demonstrates:
- `type: wait` agents (pure `asyncio.sleep`, cross-platform — no shell `sleep` dependency)
- Templated `duration` (`"{{ workflow.input.poll_interval_seconds }}s"`)
- Loop-back from wait → script for polling
- `reason` field surfaced in the dashboard

```bash
conductor run examples/wait-step.yaml \
--input poll_interval_seconds=2 --input max_attempts=3
```

### wait-smoke.yaml

Minimal wait-only workflow — no LLM, no scripts, no provider required.
Useful as a smoke test for installation, dashboard rendering, and
interrupt/timeout behavior. Three sequential wait steps demonstrate
the three duration syntaxes (suffixed string, templated, plain numeric).

```bash
conductor run examples/wait-smoke.yaml

# Watch wait nodes animate in the dashboard:
conductor run examples/wait-smoke.yaml --web

# Trigger the workflow timeout cancelling an in-flight wait:
conductor run examples/wait-smoke.yaml --input middle_duration_ms=10000
```

## Planning and Implementation

### plan.yaml
Expand Down
92 changes: 92 additions & 0 deletions examples/wait-smoke.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Wait Smoke Test
#
# Minimal `type: wait` workflow with no LLM calls, no scripts, no
# provider dependency. Useful for:
#
# - Verifying conductor installs and runs without configuring a provider.
# - Quickly sanity-checking dashboard rendering of wait nodes (with --web).
# - Confirming Esc / Ctrl+G cancels an in-progress wait immediately.
# - Confirming workflow-level limits.timeout_seconds cancels a long wait.
#
# Three wait steps run sequentially, each demonstrating a different
# duration syntax (suffixed string, templated workflow.input, plain
# numeric). All three are short by default so the workflow finishes in
# under two seconds.
#
# Usage:
# conductor run examples/wait-smoke.yaml
#
# # Watch wait nodes animate in the dashboard:
# conductor run examples/wait-smoke.yaml --web
#
# # Override the templated middle wait:
# conductor run examples/wait-smoke.yaml --input middle_duration_ms=750
#
# # Press Esc during the workflow to cancel the in-flight wait:
# conductor run examples/wait-smoke.yaml --input middle_duration_ms=30000
#
# # Watch the workflow timeout cancel an in-flight wait:
# conductor run examples/wait-smoke.yaml --input middle_duration_ms=30000
# # → workflow.limits.timeout_seconds (3) fires before middle finishes.

workflow:
name: wait-smoke
description: Minimal wait-only workflow for smoke testing and demos
version: "1.0.0"
entry_point: first

# No runtime block: wait steps don't call a provider, so this workflow
# runs without any provider configuration. The CLI defaults to
# `copilot` for parsing purposes, but no API calls are made.

input:
middle_duration_ms:
type: number
default: 500
description: Duration of the middle wait step, in milliseconds.

limits:
# Each wait step counts as one iteration (it's a step, not an LLM
# call). Three waits + headroom for routing = 5.
max_iterations: 5
# If the templated middle_duration_ms is set very high, the workflow
# timeout will fire and cancel the in-flight wait — useful for
# exercising the timeout path.
timeout_seconds: 3

agents:
# Suffixed string duration — the most common form.
- name: first
type: wait
description: First wait — fixed suffixed-string duration.
duration: 200ms
reason: First pause (suffixed-string duration)
routes:
- to: middle

# Templated duration — reads from workflow.input. Renders locally
# (no LLM), so workflow.input is available without an explicit
# `input:` declaration.
- name: middle
type: wait
description: Middle wait — duration templated from workflow.input.
duration: "{{ workflow.input.middle_duration_ms }}ms"
reason: Middle pause (templated duration)
routes:
- to: last

# Plain numeric duration — interpreted as seconds.
- name: last
type: wait
description: Last wait — plain numeric duration in seconds.
duration: 0.3
reason: Final pause (plain numeric seconds)
routes:
- to: $end

# Workflow output exposes each step's waited_seconds so callers can
# verify the contract end-to-end.
output:
first_waited: "{{ first.output.waited_seconds }}"
middle_waited: "{{ middle.output.waited_seconds }}"
last_waited: "{{ last.output.waited_seconds }}"
Loading
Loading