feat(daemon): isolate provider startup recovery by galligan · Pull Request #119 · outfitter-dev/dispatch

galligan · 2026-09-12T16:53:36Z

Context

The registry and control plane must remain readable while one provider is starting, recovering, or unavailable. Provider lifecycle failures should remain local, while shared registry or control failures stay visible as daemon failures.

What changed

Starts the registry and control service before provider workers and exposes provider-local readiness snapshots and diagnostics.
Adds independently supervised provider workers with generation-aware recovery, reactor handling, and owned-process cleanup.
Keeps cached status, receipt, model, and plain-text planning reads bounded and truthful during provider startup or outage.
Fences server-request and stale-route actions by generation, closes late connections during shutdown, and preserves the legacy safe-read compatibility floor.

Verification

just check passed Ruff, formatting, strict mypy, 1,345 tests with 17 live tests deselected, wheel and sdist builds, and package-content validation at the current stack tip.
The focused provider, recovery, and compatibility suite passed 99 tests.
Independent targeted review scored 5/5 with zero P0–P2 findings.
Final exact-head composed-stack review at 7e5d6ba is clean at 5/5 with zero P0–P3 findings; adversarial supervisor coverage exercised all 27 immediate and 12 delayed recovery/closure outcome combinations.
Hosted current-head check and Bootstrap smoke (macOS) are successful.

Risks and boundaries

Provider actions remain unavailable or in startup state until a current generation is ready; preview paths do not start a Hermes runtime. The stack remains ready for review and unmerged.

Closes: DIS-83

linear-code · 2026-09-12T16:53:40Z

DIS-83

galligan · 2026-09-12T16:54:02Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

cursor · 2026-09-13T19:28:07Z

PR Summary

High Risk
Touches daemon startup, provider reconnect/generation fencing, and shared registry recovery paths that can terminate the whole process on failure.

Overview
Restructures dispatchd so the registry and control socket come up before Codex connects, with a new ProviderManager supervising per-binding workers, publishing readiness in status, and treating SharedCoreFailure as daemon-fatal while ordinary provider crashes stay local and retryable.

Provider I/O is fenced by connection generation: Ctx drops the always-on LaneClient, renames session id to connection_generation, and routes call recheck() against live router availability (plus recheck_generation for server-request responses) instead of passing generation through every handler.

Cached reads stay honest during outages: models --no-refresh uses registry catalog plus new model_config storage (schema v28); new-plan/LaunchPlan report provider readiness without launching; legacy compatibility drops changed status from pre-handshake safe ops.

The supervisor races reactor, recovery, and disconnect per generation, wraps registry recovery in shared-core errors, and _spawn_client cleans up partial startup on failure.

^{Reviewed by Cursor Bugbot for commit f969fb2. Configure here.}

cursor · 2026-09-13T19:28:08Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_dfc26489-8fae-4d9f-be23-88bcfebed260)

chatgpt-codex-connector · 2026-09-13T19:28:15Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-13T22:07:53.446429Z	`bdb5218`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 722a51278c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cursor · 2026-09-13T20:33:46Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_61eee65f-2b49-4a1c-afc4-4fda7aa2c907)

galligan · 2026-09-13T20:40:04Z

@codex review

chatgpt-codex-connector · 2026-09-13T20:45:52Z

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 2101434b0b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

galligan · 2026-09-13T20:49:36Z

@codex review

chatgpt-codex-connector · 2026-09-13T20:55:33Z

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 65a7a3c6bb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cursor · 2026-09-13T22:02:36Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_bd3056c1-22fe-4efb-8a12-cd514a0f438e)

galligan · 2026-09-13T22:03:23Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bdb5218a90

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

galligan · 2026-09-13T23:23:15Z

Merge activity

Sep 13, 11:23 PM UTC: A user started a stack merge that includes this pull request via Graphite.
Sep 13, 11:40 PM UTC: Graphite rebased this pull request as part of a merge.
Sep 13, 11:40 PM UTC: @galligan merged this pull request with Graphite.

cursor · 2026-09-13T23:39:38Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_693cd3e0-475d-44f1-b926-6dd139fadeeb)

This was referenced Sep 12, 2026

feat(hermes): add correlated local adapter #120

Merged

fix(hermes): quarantine replaced session generations #121

Merged

feat(hermes): add operator diagnostics and package proof #122

Merged

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from ae914b9 to 722a512 Compare September 13, 2026 18:20

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from e153394 to 206e948 Compare September 13, 2026 18:20

galligan marked this pull request as ready for review September 13, 2026 19:28

chatgpt-codex-connector Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread src/outfitter/dispatch/daemon/supervisor.py

Comment thread src/outfitter/dispatch/core/handlers.py Outdated

Comment thread src/outfitter/dispatch/daemon/supervisor.py

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from 722a512 to cf65364 Compare September 13, 2026 20:33

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from 206e948 to 2e1b5ca Compare September 13, 2026 20:33

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from 2e1b5ca to bedb1a6 Compare September 13, 2026 20:37

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from cf65364 to 2101434 Compare September 13, 2026 20:37

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from 2101434 to 65a7a3c Compare September 13, 2026 20:48

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from bedb1a6 to 53d321f Compare September 13, 2026 20:48

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from 65a7a3c to bdb5218 Compare September 13, 2026 22:02

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from 53d321f to 68f04b0 Compare September 13, 2026 22:02

chatgpt-codex-connector Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread src/outfitter/dispatch/daemon/supervisor.py Outdated

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from bdb5218 to 67fd6b8 Compare September 13, 2026 22:32

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from 68f04b0 to d90c934 Compare September 13, 2026 22:32

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from 67fd6b8 to b0609f4 Compare September 13, 2026 22:50

galligan force-pushed the dis-82-extract-shared-provider-observations-and-receipt-transitions branch from d90c934 to 64ca4a2 Compare September 13, 2026 22:50

galligan changed the base branch from dis-82-extract-shared-provider-observations-and-receipt-transitions to graphite-base/119 September 13, 2026 23:37

galligan changed the base branch from graphite-base/119 to main September 13, 2026 23:38

galligan added 7 commits September 13, 2026 23:39

feat(daemon): isolate provider startup recovery

8fe64a5

fix(daemon): preserve fatal and cached provider state

1daad2c

fix(daemon): withdraw disconnected recovering clients

253435d

fix(daemon): preserve fatal recovery failures

3d5f5f5

fix(daemon): prioritize shared-core failures

429cfe3

fix(daemon): escalate reactor registry failures through exception groups

4dbb8bf

test(core): use connection generation in server-request binding test

f969fb2

galligan force-pushed the dis-83-make-provider-startup-and-recovery-independent-within branch from b0609f4 to f969fb2 Compare September 13, 2026 23:39

galligan merged commit 76ed777 into main Sep 13, 2026
9 checks passed

Conversation

galligan commented Sep 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

What changed

Verification

Risks and boundaries

Uh oh!

linear-code Bot commented Sep 12, 2026

Uh oh!

galligan commented Sep 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

cursor Bot commented Sep 13, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot commented Sep 13, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

galligan commented Sep 13, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026

Uh oh!

galligan commented Sep 13, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 13, 2026

Uh oh!

cursor Bot commented Sep 13, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

galligan commented Sep 13, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

galligan commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

cursor Bot commented Sep 13, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

galligan commented Sep 12, 2026 •

edited

Loading

galligan commented Sep 12, 2026 •

edited

Loading

cursor Bot commented Sep 13, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Sep 13, 2026 •

edited

Loading

galligan commented Sep 13, 2026 •

edited

Loading