Skip to content

Python: HITL respond-URL addressing from inside workflows#7001

Open
ahmedmuhsin wants to merge 8 commits into
microsoft:mainfrom
ahmedmuhsin:feature/python-durabletask-workflow-hitl-context
Open

Python: HITL respond-URL addressing from inside workflows#7001
ahmedmuhsin wants to merge 8 commits into
microsoft:mainfrom
ahmedmuhsin:feature/python-durabletask-workflow-hitl-context

Conversation

@ahmedmuhsin

Copy link
Copy Markdown
Contributor

Motivation & Context

The durable function HITL flow pauses a workflow with request_info and waits for a human to POST a response to the /respond endpoint. The gap was that nothing inside the workflow could build that respond URL. An executor only receives its message payload, not the orchestration instanceId, and the requestId does not exist until request_info runs. So to email someone an approval link you had to pull both ids from the HTTP run response or poll the status endpoint from outside the app.

This came out of a real POC where someone wanted Agent B to email a reviewer when it finished, then resume to Agent C once approved, but had no way to address the running workflow from within it. This change lets a workflow build its own respond URL and notify a human from inside the graph with no caller plumbing of instanceId or requestId, and it works for nested sub workflows too.

Description & Review Guide

  • What are the major changes?

    • The orchestrator now surfaces orchestration metadata (the addressable root instanceId, the workflow name, and for nested workflows the request path prefix) onto each executor through the durable runner context. There is no new core API.
    • A new WorkflowHitlContext helper in agent-framework-azurefunctions builds the canonical respond and status URLs from that metadata. It returns None in process so the same executor degrades gracefully when it is not on the durable host.
    • WorkflowHitlContext.pending_request_id(ctx) reads the id request_info just generated back off the context, so the user never mints an id by hand. It works on any host through the existing core runner context protocol.
    • For nested sub workflows the address context propagates down call_sub_orchestrator via a new SUBWORKFLOW_ADDRESS_KEY marker, so an executor at any depth builds a URL that targets the top level instance with a qualified request id. That marker is stripped from untrusted input alongside the existing input marker, so a top level caller cannot forge it.
    • Samples 12 and 13 show the two step pattern. The review executor reads the request id back and sends it to a downstream notify executor that builds the URL, so a retried activity never emails a dead link because only the committed attempt reaches the notifier.
  • What is the impact of these changes?

    • Additive. No core change, nothing public removed, and top level behavior is unchanged when the helper is not used.
    • New unit tests cover the metadata round trip, the id read back, the nested ordinal and prefix agreement with the read side, marker stripping, and URL building. Integration tests assert that the URL the helper builds equals the server respondUrl and that posting to it resumes the run, for both the flat (12) and nested (13) samples.
  • What do you want reviewers to focus on?

    • The per executor ordinal used to build the nested prefix at dispatch has to match the enumerate index the status and respond endpoints use when they resolve a nested request. That agreement is the one spot where a mistake would send an emailed URL to the wrong child or a 404.

Related Issue

No separate issue. This builds on the Durable Task multi-workflow and sub-workflow hosting work that landed on main in #6696, and now targets main directly.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI review requested due to automatic review settings July 8, 2026 21:59
@giles17 giles17 added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Jul 8, 2026
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/azurefunctions/agent_framework_azurefunctions
   _app.py58415174%82–83, 86–89, 317, 375, 417–418, 422–423, 425–427, 435–437, 439, 442, 452–453, 455–457, 459, 463, 466, 468, 470, 473, 492–494, 497–500, 506, 508, 510–511, 513, 533–535, 537, 543–544, 551, 553, 568–573, 586, 588, 603–604, 606–607, 612–614, 616–619, 623, 628–631, 635, 641, 694, 731, 737, 740, 743, 888–889, 997, 1005–1006, 1026–1028, 1034–1036, 1042–1044, 1077–1078, 1138–1139, 1188–1189, 1194, 1276, 1279, 1288–1290, 1292–1294, 1296, 1298, 1309–1312, 1314, 1316–1317, 1319, 1326–1327, 1329–1330, 1332–1333, 1335, 1339, 1349–1351, 1353–1354, 1356–1358, 1365, 1367–1368, 1370, 1391, 1396, 1408, 1480, 1570, 1585–1588, 1613
   _hitl_context.py61198%192
   _routes.py46393%57, 60, 63
packages/durabletask/agent_framework_durabletask/_workflows
   activity.py741777%62, 85–91, 110, 149–153, 159, 172–173
   orchestrator.py45723648%209–211, 263–265, 283, 300–302, 341–342, 344–351, 353, 360, 375, 377–386, 388–389, 391, 488–489, 491–492, 494–497, 500–502, 504–505, 507–512, 514–517, 519, 521–525, 534–536, 538, 540–541, 543–548, 550–551, 553–554, 556–557, 559, 572–576, 583, 596, 603–605, 607–608, 610, 628–629, 650, 745, 749, 787, 793, 818, 822, 828–835, 837–838, 897, 929–930, 933–934, 944–945, 947–950, 959, 1021, 1024–1025, 1032, 1039, 1043, 1056, 1058–1061, 1063, 1066–1074, 1076, 1083–1085, 1089–1092, 1098–1100, 1102, 1106, 1108–1110, 1113, 1120–1124, 1127–1129, 1135–1139, 1141–1148, 1152–1153, 1155–1157, 1160–1164, 1166–1168, 1171–1172, 1175–1176, 1179, 1181, 1186–1187, 1190–1191, 1193, 1207, 1213–1214, 1216–1217, 1224–1229, 1235–1237, 1242, 1244–1245, 1250, 1252, 1254, 1258–1259, 1263–1265
   runner_context.py812371%59–60, 68, 73, 85, 88, 93, 96, 99, 110, 113, 116, 121, 124–127, 130, 133, 162, 167–168, 171
   serialization.py101496%243, 358–360
TOTAL44595521588% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
9046 33 💤 0 ❌ 0 🔥 2m 33s ⏱️

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 91% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes, Design Approach


Automated review by ahmedmuhsin's agents

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables Python Durable Functions workflows (including nested sub-workflows) to construct canonical HITL /respond (and /status) URLs from inside executors, by surfacing orchestration addressing metadata onto the runner context and adding a small helper (WorkflowHitlContext) in agent-framework-azurefunctions.

Changes:

  • Surface durable-host orchestration metadata (root instance/workflow + nested request-path prefix) to executors via durabletask runner context host_metadata.
  • Add WorkflowHitlContext helper to build respond/status URLs and to read back the latest request_info-generated request id from the runner context.
  • Update samples + add unit/integration tests, including nested sub-workflow address propagation and marker-stripping security coverage.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/samples/04-hosting/azure_functions/13_subworkflow_hitl/README.md Documents in-subworkflow notification pattern using WorkflowHitlContext.
python/samples/04-hosting/azure_functions/13_subworkflow_hitl/function_app.py Adds inner-workflow notifier executor and wires it into the nested HITL sample.
python/samples/04-hosting/azure_functions/12_workflow_hitl/README.md Documents in-workflow reviewer notification + respond URL building.
python/samples/04-hosting/azure_functions/12_workflow_hitl/function_app.py Adds notifier executor and routes approval result explicitly to publish step.
python/packages/durabletask/tests/test_workflow_serialization.py Adds tests ensuring subworkflow address marker is stripped from untrusted input.
python/packages/durabletask/tests/test_workflow_activity.py Adds tests asserting host_context is surfaced as runner_context.host_metadata.
python/packages/durabletask/tests/test_subworkflow_orchestration.py Adds tests for address resolution + per-superstep subworkflow ordinal/prefix agreement.
python/packages/durabletask/agent_framework_durabletask/_workflows/serialization.py Introduces SUBWORKFLOW_ADDRESS_KEY and strips it alongside SUBWORKFLOW_INPUT_KEY.
python/packages/durabletask/agent_framework_durabletask/_workflows/runner_context.py Adds host_metadata plumbing to the durable activity runner context.
python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py Threads address context into activity/subworkflow dispatch and propagates nested prefix.
python/packages/durabletask/agent_framework_durabletask/_workflows/activity.py Reads host_context and sets it on the runner context for executor access.
python/packages/core/agent_framework/azure/init.pyi Re-exports WorkflowHitlContext from the agent_framework.azure facade typings.
python/packages/core/agent_framework/azure/init.py Adds lazy export mapping for WorkflowHitlContext.
python/packages/azurefunctions/tests/test_hitl_context.py New unit tests for WorkflowHitlContext URL building/base URL resolution/pending id.
python/packages/azurefunctions/tests/integration_tests/test_13_workflow_subworkflow_hitl.py Integration test asserts helper-built URL matches server qualified respondUrl (nested).
python/packages/azurefunctions/tests/integration_tests/test_12_workflow_hitl.py Integration test asserts helper-built URL matches server respondUrl (flat).
python/packages/azurefunctions/agent_framework_azurefunctions/_hitl_context.py New helper implementation for HITL URL construction + pending request id read-back.
python/packages/azurefunctions/agent_framework_azurefunctions/init.py Exports WorkflowHitlContext from the package public surface.

Comment thread python/samples/04-hosting/azure_functions/12_workflow_hitl/README.md Outdated
Comment thread python/samples/04-hosting/azure_functions/13_subworkflow_hitl/README.md Outdated
Comment thread python/packages/azurefunctions/tests/integration_tests/test_12_workflow_hitl.py Outdated

@larohra larohra left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few notes on the HITL addressing mechanism -- 2 likely must-fix URL-correctness bugs (hardcoded route prefix, loopback scheme), 2 cheap de-risking refactors (respond/status URL dedup, the cross-package host_context key contract), and 1 optional hardening note on the sub-workflow ordinal. Details inline. Nothing here blocks the durable use case; the first two just affect whether generated links resolve in non-default setups.

Comment thread python/packages/azurefunctions/agent_framework_azurefunctions/_hitl_context.py Outdated
Comment thread python/packages/azurefunctions/agent_framework_azurefunctions/_hitl_context.py Outdated
Comment thread python/packages/azurefunctions/agent_framework_azurefunctions/_hitl_context.py Outdated
@ahmedmuhsin ahmedmuhsin marked this pull request as ready for review July 13, 2026 19:18
@ahmedmuhsin ahmedmuhsin requested a review from a team as a code owner July 13, 2026 19:18

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 89% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes, Design Approach


Automated review by ahmedmuhsin's agents

@kshyju kshyju left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thorough review of all 18 files. The architecture is clean — address context propagation through the orchestration tree, security-conscious marker stripping, and the retry-safe two-step notify pattern are all well done. The critical per-executor ordinal / enumerate agreement is correct and well-tested. LGTM.

Minor (non-blocking): In _hitl_context.py ~L192, the comment says "An override may already include a scheme" but at that point base_url_override is already falsy (returned early). The comment should reference WEBSITE_HOSTNAME instead, e.g.: "WEBSITE_HOSTNAME may include a scheme (unusual but possible); otherwise infer one (http for local loopback, https otherwise)."

@kshyju

kshyju commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Thorough review of all 18 files. The architecture is clean — address context propagation through the orchestration tree, security-conscious marker stripping, and the retry-safe two-step notify pattern are all well done. The critical per-executor ordinal / enumerate agreement is correct and well-tested. LGTM.

Minor (non-blocking): In _hitl_context.py ~L192, the comment says "An override may already include a scheme" but at that point base_url_override is already falsy (returned early). The comment should reference WEBSITE_HOSTNAME instead, e.g.: "WEBSITE_HOSTNAME may include a scheme (unusual but possible); otherwise infer one (http for local loopback, https otherwise)."

From GHCP review ^

@ahmedmuhsin

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. I fixed the base_url comment in 348da78 to reference WEBSITE_HOSTNAME instead of an override, since the override returns early above.

…ecutors

Let a workflow notify a human reviewer (e.g. email an approval link) from inside the
graph, without the caller threading the instanceId/requestId by hand.

- durabletask: the orchestrator injects host_context {instance_id, workflow_name,
  request_path_prefix} into each activity input; CapturingRunnerContext surfaces it as
  host_metadata. No new core API.
- azurefunctions: WorkflowHitlContext.from_context(ctx) builds the canonical
  respond/status URLs (returns None in-process so callers degrade gracefully).
  Re-exported through the agent_framework.azure lazy namespace.
- Nested sub-workflows: the address context (root instance + workflow name +
  accumulated {executor}~{ordinal}~ prefix) propagates down call_sub_orchestrator via a
  new SUBWORKFLOW_ADDRESS_KEY marker, so an executor at any depth builds a URL that
  targets the addressable top-level instance with a qualified request id. The per-child
  ordinal matches the read-side enumerate() index used by the status/respond endpoints.
  The marker is stripped from untrusted input alongside SUBWORKFLOW_INPUT_KEY
  (confused-deputy / info-leak guard).
- Samples 12 and 13 reworked into the retry-safe two-step notify pattern: the emitter
  generates an explicit request id and a downstream NotifyExecutor builds the URL and
  notifies, so failed upstream retries never produce a dead link.

Tests: unit coverage for the metadata round-trip, address/ordinal agreement (fan-out at
depth and nested prefix accumulation), marker stripping, and URL building; integration
tests assert the helper-built URL equals the server respondUrl and resumes the run, for
both the flat (12) and nested (13) samples.
…g one in samples

Add WorkflowHitlContext.pending_request_id(ctx), an async helper that returns the
id request_info just generated (read from the runner context's pending request-info
events). This works on any host via the core RunnerContext protocol method, so it
needs no core change.

Samples 12 and 13 now call request_info() and read the id back to forward to the
NotifyExecutor, instead of minting a uuid by hand and passing request_id=. The
read-back happens in the same activity execution that generated the id, so the
pending request event and the notify message still commit together with the same id
(retry-safe; failed upstream retries notify no one).
…afety

Tighten pending_request_id docstring to require calling it immediately after request_info, and explain why that is safe on the durable host (each executor runs in its own activity with its own runner context, so the pending set only holds this executor's requests and the newest is the one just emitted). Document the two-step notify pattern in the 12 and 13 sample READMEs, including the downstream-notifier retry safety and the nested address-prefix propagation.
- test_subworkflow_orchestration: replace the mypy-only type:ignore[arg-type] with a cast so the ty checker passes too (the other four checkers already honored the ignore).

- samples 12/13 README: guard the notify snippet against None before build_respond_url to match the documented graceful-degradation behavior.

- integration tests 12/13: reword comments that implied request_info now generates an explicit uuid4; it generates the id internally by default.
…TL PR review

- Resolve the route prefix from host.json (extensions.http.routePrefix, default api) in a new azurefunctions _routes module, used by both the server endpoints and WorkflowHitlContext, so a custom or empty routePrefix no longer 404s respond/status URLs (was hardcoded /api/ in four places).

- Extract respond/status URL construction into one shared builder called from _app.py and _hitl_context.py, removing the sync-by-test duplication.

- Broaden loopback detection (localhost, 127.0.0.0/8, 0.0.0.0, ::1, [::1]) via a _is_loopback helper so local links use http.

- Pin the host_context key names as shared constants in durabletask so producer and azurefunctions consumer cannot drift.

- Reword a stale base_url comment to reference WEBSITE_HOSTNAME.

- Add unit tests for the route module and loopback handling.
…st URL

The run and status endpoints now derive the base URL and route prefix from the incoming request URL (the value the host actually routed) via split_request_url, so the caller-visible respond/status URLs no longer depend on reading host.json on the server. The in-workflow helper keeps reading host.json since it has no request context. Replaces strip_route_prefix and updates its tests.
Extract the read-side subworkflows grouping into a shared _index_subworkflows helper (used by the orchestrator) and add test_readside_index_matches_dispatch_ordinal, which round-trips a fan-out through that helper and asserts subworkflows[executor][ordinal] resolves to the child the dispatch stamped that ordinal onto. Turns the previously comment-only write-ordinal / read-index invariant into a shared, CI-enforced one.
@ahmedmuhsin ahmedmuhsin force-pushed the feature/python-durabletask-workflow-hitl-context branch from 33b4b10 to 82ed863 Compare July 15, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants