Skip to content

subordinate_status and the console still read the raw status column — a supervisor waits on an agent the platform knows has stopped #589

Description

@serge-ivo

The health verdict reaches 2 of 5 loop-run readers; the three it misses are the supervisor and the owner

f01d5b4 (#580 AC3) added runHealth so a client would stop inferring liveness. It has exactly two production consumers (grep -rn "runHealth\|isStalled(\|waitClause" workers/api/src workers/mcp/src store | grep -v test → 8 hits, 2 call sites): routes/tools.ts:1046 and work-report.ts:203.

The five readers of agent_loop_runs:

reader applies verdict?
routes/tools.ts:1055,1063 (/loop) yes
agent-think.ts:451describeLoopRun yes
tool-registry.ts:231,247 (check_work) yes
connectors/supervision.ts:714 (check_delegation) no
lib/instance-work.ts:223subordinate_status no
store/console/src/lib/workInFlight.ts:29 + tabs/LoopRunsSection.tsx:96 no

Three answers, one run, same instant

For a run parked 4.35 h on an engine limit:

  • check_instance_loophealth:"waiting", waitNote:"WAITING, not stalled and not working…"
  • subordinate_statusactivity:"working", quietForMinutes: 261
  • console "Autonomous runs" (LoopRunsSection.tsx:152) → step 1/40 · started 4h ago + a Stop button

Mechanism

  • subordinate-payload.ts:177 and :228activity = runs.some((r) => r.status === "running") ? "working" : "idle", the raw column.
  • subordinate-observation.ts:152quietForMinutes: minutesSince(r.lastProgressAt ?? r.startedAt, now), the column agent-loop-store.ts:69-74 explicitly calls "NOT a liveness signal, and it was read as one for two releases".
  • instance-work.ts:233-234 does not SELECT last_alive_at, waiting_reason, waiting_until — the supervision path cannot produce a verdict.

The description sells the wrong surface hardest (supervision.ts:550): "What every agent you supervise is doing RIGHT NOW… 'which ones are idle' is roster[].activity."

Two comments are now false

  • run-sweeper.ts:67"the same rule summarizeSubordinates uses, so the sweeper and the supervisor still agree on 'quiet'". The sweeper uses COALESCE(last_alive_at, last_progress_at, started_at) (:70); summarizeSubordinates uses lastProgressAt ?? startedAt.
  • workInFlight.ts:28"A run is live only while the server says running; every terminal state is a different word." Since fd1c323, running covers working, waiting and stalled.

Why this misleads into a wrong action

run-sweeper.ts:9-11 already names the victim: "subordinate_status reports it as work in flight, so a supervisor keeps waiting on an agent that stopped hours ago." A Coder Lead blocks on a dead subordinate; the owner sees a Stop button for a run that stopped itself.

Acceptance criteria

  1. Every reader of agent_loop_runs applies the same verdict, or explicitly declines to make a liveness claim. subordinate-payload.ts:177 is the single highest-value line.
  2. instance-work.ts's SELECT carries the columns a verdict needs.
  3. A guard asserts every reader of that table derives health from one implementation — denominator stated per ADR 0002 as readers-found, not a hand-listed set. Two surfaces deriving one fact independently is the A run whose engine died at step 1 reported 'running' for 4.35 hours with a fresh lastProgressAt, and wrote nothing to the trace #580 defect; this is the same defect at a different layer.
  4. The two false comments are corrected or deleted — a comment asserting an agreement that no longer holds is worse than none.
  5. Red demonstrated: a parked run must not be reported as working by subordinate_status.

Verified vs inferred

  • Verified: the grep and its counts; all six reader call sites; the three live answers; every quoted comment.
  • Inferred: nothing load-bearing.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions