Skip to content

feat: add output field constraints (enum, pattern, range, length, optional, nullable) - #372

Open
hertznsk wants to merge 9 commits into
microsoft:mainfrom
hertznsk:feat/output-field-constraints
Open

feat: add output field constraints (enum, pattern, range, length, optional, nullable)#372
hertznsk wants to merge 9 commits into
microsoft:mainfrom
hertznsk:feat/output-field-constraints

Conversation

@hertznsk

@hertznsk hertznsk commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Workflow authors can now declare real constraints in their agents' output: blocks — allowed value lists (enum), regex patterns (pattern), numeric ranges (minimum/maximum), string length limits (minLength/maxLength), optional fields (required: false), and nullable fields (nullable: true). Every provider (Claude, Copilot, Hermes, Claude Agent SDK, ACA) enforces them the same way, and on Claude the model gets an automatic in-session chance to fix a constraint violation before the workflow fails.

What changed

  • OutputField schema model gains eight optional fields (enum, pattern, minimum, maximum, minLength, maxLength, required, nullable) with Pydantic validators rejecting type-inappropriate combinations (e.g. pattern on type: number, None entries in enum — use nullable: true instead).
  • Shared schema builders (_schema.py) emit the new keywords in both flavours (JSON schema and prompt schema); required arrays list only required fields; nullable renders as type: [<T>, "null"]; no default keys ever appear in generated schemas.
  • Pydantic-ai converter (Claude path) enforces constraints via AfterValidator (strict Python semantics identical to the final validator — no Literal[...] / Rust-regex surprises) while advertising them to the model via json_schema_extra. Constraint violations raise inside the pydantic-ai output-retry loop, giving the model a free in-session self-correction. Optional fields use exclude_unset=True so an omitted key is never materialized as None.
  • Strict validator (validate_output) checks all eight constraints after the type check, recursion-aware (object properties and array items). Legacy error messages remain byte-identical when no new fields are used.
  • YAML-load gate: root-level required: false is rejected at load time via a WorkflowConfig model validator (optional output fields are only allowed inside object properties).
  • Provider parity tests prove one shared YAML schema produces the same keywords in Copilot/Hermes prompt schemas, the claude_agent_sdk output_format payload, pydantic constraints on the Claude dynamic model, and identical ValidationError from validate_output. The ACA wire boundary is covered too (constraint fields survive the round trip).
  • Docs: docs/workflow-syntax.md gains a "Field constraints" subsection; new example examples/output-constraints.yaml exercising every field; CHANGELOG entry under Unreleased.

Deliberately out of scope

  • No additionalProperties: false / extra-key rejection (deferred to a follow-up).
  • No default values for optional fields — an omitted optional key stays omitted.
  • No new JSON Schema keywords beyond the agreed eight (no exclusive bounds, multipleOf, format, const, etc.).
  • Workflows that do not use the new fields behave byte-identically (golden schema tests unchanged).

Test plan

  • uv run pytest -m "not performance" -q — full suite green
  • make lint / make typecheck — clean
  • make validate-examples — green
  • New unit tests cover every rejection combination, parity across providers, and the Claude in-session retry path (via a FunctionModel returning a violating payload then a valid one)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant