Skip to content

US-45.3: review dispatch, checkpoint-bound findings and a server-side round ceiling - #905

Closed
mkreyman wants to merge 3 commits into
masterfrom
feature/us-45.3-review-dispatch
Closed

mkreyman wants to merge 3 commits into
masterfrom
feature/us-45.3-review-dispatch

Conversation

@mkreyman

Copy link
Copy Markdown
Owner

US-45.3 (Epic 45, change threads, #882): review as its own dispatch, checkpoint-bound findings, fixes, and the round ceiling computed from the thread.

First slice. AC-45.3.8 (runner contract 1.21.0: review in dispatchable_kinds, the review payload and messages on the wire) is NOT in this PR. Publishing review as dispatchable needs loopctl to actually send it (a review dispatcher alongside TriageDispatcher) and channel handlers for finding/verdict/fix messages, attributed to the runner's ledger row the way this PR attributes them to the minted key. That is its own change. The review gate is still running on this PR.

How review authorship is proven

#901 inferred the judge from the calling key and was circumvented three times. Nothing here infers it.

  • Loopctl.Threads.Reviews.place/4 MINTS the reviewer's dispatch and records it in a new thread_reviews table (story, dispatch, agent, checkpoint, round).
  • A finding or verdict is accepted only when Dispatches.dispatch_for_api_key/2 resolves the calling key to that exact dispatch row for this story. No lineage walk, no agent comparison. Everything else is 403 review_dispatch_required.
  • The entry is bound server-side to the review and the checkpoint it was placed to read.

What it does

  • Placement (POST /stories/:id/thread/reviews, orchestrator+, human anchor, halt-checked in the context like Placement):
    • the review dispatch is a SIBLING of implementer_dispatch_id (same parent), so it is never on the implementer's chain;
    • the caller must hold that parent in its lineage (DispatchController's ceiling, including its operator and legacy-unlineaged cases) and must not be the implementer or below it;
    • its agent must not be the claimant nor a principal that recorded a checkpoint of the thread (rechecked on every judgement, since a claim can move);
    • the raw key is returned once.
  • Findings carry severity (critical/high/medium/low), optional location, and introduced_by: refused in round 1, required after it, a checkpoint of the story at or before the reviewed one or none, stored canonicalised.
  • Verdicts: one per review (code check plus a partial unique index). A review placed for round N completes it only while N-1 rounds are complete (review_round_superseded). The verdict revokes the review's dispatch, freeing its agent's single agent-role key slot for the next round.
  • Fixes (exact_role: :agent): the checkpoint fence (claimant, epoch, live lease); a checkpoint of the current claim recorded after every checkpoint its findings were found in; findings of completed rounds only.
  • Ceiling: rounds 1 and 2 always; round 3 only when a round-2 finding's introduced_by names a checkpoint carrying a fix; never round 4. When a round reaches the ceiling with a material finding, the verdict writes a review_ceiling escalation entry in its transaction, then escalates the delivery stage (best-effort, logged).
  • Payload (GET /stories/:id/thread/reviews/:review_id): story and criteria, the checkpoint with the branch and parent commit, the latest entries, every fix with its findings, the rounds.
  • Refusal codes are all new and none is self_review_blocked.
  • MCP 2.106.0: thread_place_review, thread_review_get, thread_finding, thread_verdict, thread_fix. Findings and verdicts travel only on LOOPCTL_API_KEY, never the agent key.
  • Migration 20260926160000: thread_reviews (RLS enabled) and nullable judgement columns on thread_entries with a shape CHECK. No manual step.

Design decisions

  • thread_reviews.dispatch_id has no FK, like thread_entries.dispatch_id: dispatches are on AdminRepo.
  • A verdict takes no outcome field. Whether a round asked for changes is its findings.
  • An implementer parent that is revoked or expired cannot parent the sibling: 409 review_parent_inactive. With orchestrator dispatches capped at four hours, a long-running story can hit this; the remedy is a fresh orchestrator dispatch under the same root, not a weaker placement.
  • place/4 is not idempotent (it mints a credential). A retried placement makes a second review for the same round, and only one of them can complete it.

Checks

  • Context tests: 35. Controller tests: 6. Plus the existing thread, tier-capability, context-surface, custody-surface, openapi and route-snapshot tests, all green. The node suite is green, route coverage included.
  • The tests are async: false and COMMITTED, as PlacementTest is. A sandboxed thread write holds the tenant's audit-chain lock, and the AdminRepo mint then waits on it for ever.

Mutations (bin/mutate.sh), exit 0 = caught

id what the mutation broke exit
M01 authority: review looked up by the minting dispatch 0
M02 authority: review is THIS story's 0
M03 placer not on implementer chain 0
M04 parent within caller lineage 0
M05 sibling: parent is implementer parent 0
M06 root sibling operator-only 0
M07 reviewer not the claimant 0
M08 reviewer not a checkpoint recorder 0
M09 wiring: separation rechecked on judgement 0
M10 wiring: review_open on judgement 0
M11 current round equality 0
M12 wiring: current round on finding 0
M13 introduced_by refused in round 1 0
M14 introduced_by required after round 1 0
M15 introduced_by at or before found-in 0
M16 introduced_by none canonicalised 0
M17 wiring: fix claimant fence 0
M18 fix checkpoint of current claim 0
M19 fix checkpoint after findings 0
M20 fix names only known findings 0
M21 fix findings of completed rounds 0
M22 round 3 needs introduced_by a fix checkpoint 0
M23 round 3 fix must answer a round-1 finding (survived; the filter was provably unreachable and was REMOVED, see the comment in third_round_warranted?/3) 1
M24 rounds 1-2 always, then gated 0
M25 escalation only with material finding 0
M26 medium is material 0
M27 wiring: stage escalation 0
M28 wiring: verdict closes the dispatch 0
M29 wiring: halt checked on placement 0
M30 placer role orchestrator 0
M31 implementer dispatch required 0
M32 reviewer_agent_busy mapping 0
M33 agent must be this tenant's (survived: RLS on agents does the same job) 1
M34 ceiling refusal on placement 0
T01 replay compares review_id 0
T02 chain pins judgement fields 0
D01 one verdict per review index (migration) 0
C01 controller: tenant_halted is 503 0
C02 controller: exact_role agent on finding 0
C03 route: findings 0
C04 render: location_untrusted 0
C05 thread show renders review_id 0
J01 mcp: finding only on LOOPCTL_API_KEY 0
J02 mcp: place_review on orchestrator key 0
J03 mcp wiring: thread_finding dispatched 0
J04 mcp: fix on the claim key 0
J05 mcp: fix needs a finding 0
J06 mcp wiring: thread_place_review README row 0
M22b round 3 needs introduced_by a fix checkpoint, re-run after the M23 removal 0
M35 wiring: expires_in_seconds validated on placement 0

All run with --no-baseline after a green run of the same check, and </dev/null. D01 rolled the migration back and forward under the mutation; the real index was restored the same way afterwards.

Not covered by a test

  • M33: the explicit tenant_id predicate in the agent check survives mutation, because RLS on agents already hides another tenant's agent. It is kept by the repo's dual-scoping convention.
  • The placement route's role: :orchestrator plug is not mutated on its own: with it removed, the context's gate (M30) still answers 403.
  • checkpoint_id belonging to ANOTHER story of the same tenant. The test uses a random id.
  • The lock ordering, as for US-45.1.

… round ceiling

An orchestrator places a review, which mints the reviewer's dispatch as a sibling
of the implementer's; findings and verdicts are accepted only on the key that
placed dispatch minted, bound to the review and the checkpoint it reads. The
claimant records fixes naming the findings they answer. Round count and ceiling
are computed from thread_entries: round 3 only when a round-2 finding is
introduced by a round-1 fix checkpoint, never round 4, and a ceiling with a
material finding records review_ceiling and escalates the delivery stage.
Refusals have their own codes, never self_review_blocked.

Ships the MCP tools (2.106.0). The runner contract bump (AC-45.3.8) remains.
1. Ceiling: placement applied the reverse of the dispatch ceiling. Both now call
   Dispatches.lineage_within_caller?/3, the one copy; the caller must be the
   implementer's parent or an ancestor. The on-chain check it made unreachable is gone.
2. Round 3 is decided at the round-2 verdict: only fixes with a lower seq count.
3. A verdict revokes every other open review of its round; a verdict refused
   review_round_superseded revokes its own dispatch.
4. Moduledoc and MCP text: a verdict resend is 401 once the key is revoked; the
   answer-from-row path remains for a failed revocation only.
5. The ceiling escalation goes through Escalations.escalate_as_control/3, which shares
   escalate/3's transition, replay and stale_stage recovery minus the claimant check.
6. Judgement idempotency is scoped to the review, in the replay lookup and in a
   review-scoped unique index; the author-scoped index now excludes judgements.
7. LoopctlWeb.ThreadHTTP is the one render/param helper for both thread controllers;
   location_untrusted is rendered on every thread read.
8. The judgement CHECK refuses a NULL finding_ids.
9. An implementer dispatch that does not resolve is 409 unresolvable_dispatch_lineage.
10. The review payload carries the latest 100 fixes with fixes_truncated, their
    findings read by join rather than an id list.

Mutations (bin/mutate.sh, 0 = caught): 62 run, 59 caught. M33 survives (RLS on
agents), N02 survives against the dispatch controller suite (its ceiling test does
not probe the prefix direction; caught by the review suite as N01).
1. A caller with an empty lineage places a review only as the operator, decided by
   Placement.may_mint_session_dispatch/2 (now public, the one copy); a legacy key is
   409 caller_lineage_required.
2. The reviewer may not be the agent of any dispatch on the implementer's lineage or
   the review's own ancestry, checked at placement, under the record lock, and on
   every judgement write.
3. The author-scoped idempotency lookup excludes judgements, matching the partial index.
4. Placement is prepare then commit; the round and the separation are re-decided under
   the thread lock when the review is recorded.
5. review_round_superseded and reviewer_not_separate end the review and revoke its
   dispatch, for findings and verdicts.
6. The record revokes the minted dispatch on a refusal, on busy when no row landed
   (a row that did land is answered), and on a raise before re-raising.
7. The ceiling escalation takes a new control-only review_ceiling stage edge with a
   control label; the runner-reportable set is unchanged.
8. The review authority (a review of this story) is decided before the payload.
9. Reviews uses Threads.story/2 and Threads.checkpoint_of/3; its copies are gone.
10. The dispatch ceiling suite now tests the prefix direction.
11. The migration's down refuses, with an explicit Ecto.MigrationError, when judgement
    keys shared across reviews would break the per-author index it restores.
@mkreyman

Copy link
Copy Markdown
Owner Author

Closed after review round 3, the ceiling, still found material trust-path defects: the custody halt was not enforced on judgement writes, separation missed descendants of the implementer, public Threads helpers let any module write a verdict, and the ceiling's stage move was best-effort. This is the second authorship design to fail at the ceiling (after #901). The common cause is that placement minted a raw credential and handed it to the orchestrator, and each round found another hole in that credential's ceiling, separation or lifecycle. US-45.3 is being rewritten on feature/us-45.3-review-dispatch-v2: a review is a runner review dispatch loopctl places (AC-45.3.8), and judgements arrive over the runner socket attributed to the dispatch ledger row, so no credential is ever handed out. This branch stays on GitHub as the record.

@mkreyman mkreyman closed this Sep 26, 2026
mkreyman added a commit that referenced this pull request Sep 27, 2026
… 1.21.0) (#907)

* US-45.3: review a change thread as a placed runner dispatch (contract 1.21.0)

A review is a runner dispatch of kind review, placed by Placement.place_review
on the story's latest checkpoint of the current claim. Its findings and its one
verdict arrive over the runner socket as review_finding and review_verdict,
bound to the runner's accepted review-kind ledger row; no API key is minted for
or returned to anyone. Replaces the minted-reviewer-key approach of #901 and
#905.

- Threads.record_review / record_judgement / record_fix: one entry point per
  write, each re-checking its binding under the thread lock; separation
  (claimant, checkpoint author, implementer lineage chain) at placement and on
  every judgement; custody halt enforced on judgement writes.
- Rounds counted from thread_entries: one verdict per review, round 3 only when
  a round-2 finding names a checkpoint a round-1 fix carried, never round 4.
- A round-2 verdict with a material finding records a review_ceiling
  escalation; ReviewCeilingWorker moves the stage over the new review_ceiling
  edge until it lands or the claim moves.
- HTTP: POST thread/reviews (orchestrator+), GET thread/reviews/:id, POST
  thread/fixes (claimant). MCP 2.106.0: thread_request_review,
  thread_review_get, thread_fix.
- Migration 20260926160000: thread_reviews, judgement columns and CHECKs on
  thread_entries, runner_dispatches.kind widened to review.
- REVIEW_WALL_CLOCK_SECONDS / REVIEW_MAX_TURNS documented in FLY_SECRETS.

* US-45.3: review round 1 fixes on PR 907

- A judgement from a review whose claim ended (new epoch, or no claimant
  after a force-unclaim) is refused review_claim_ended, finally, and writes
  nothing, so no stale ceiling escalation; placement refuses the same.
- Rounds, the round-3 rule and the ceiling count only the current claim
  epoch's reviews, so a story claimed again starts at round 1.
- A resent verdict answers the recorded escalation and re-runs the
  idempotent slot release and ceiling enqueue.
- The verdict enqueues a unique ReviewCeilingWorker job instead of moving
  the stage inside the channel; an enqueue failure is logged, never raised.
- A fix counts toward round 3 only if the thread recorded it by the time
  round 2 was placed (thread_reviews.placed_at_seq).
- Placement runs the push pre-checks (socket, draining, kind, repo,
  subscription, free slot) before recording the review and its chain entry.
- session_ended frees a review session's slot and does nothing else;
  contract 1.21.0 text, ack and digest updated in place.
- Two placements sharing a dispatch_id on different stories answer
  dispatch_id_conflict from the unique index instead of a 500.
- OpenAPI and CHANGELOG corrected; migration down refuses while any review
  exists.
- RunnerThreadSession: the ledger read, epoch check and 422 translation
  RunnerThreads and RunnerReviews shared as copies.
- The completed rounds are read once per judgement write and reused for
  closed, superseded and the ceiling.

* US-45.3: review round 2 fixes on PR 907

- One implement-kind rule on every implement-only ledger consumer: a review
  dispatch carries the implementer's story and claim epoch, so epoch and
  story fences cannot tell it apart. DispatchLedger gains
  where_implement_kind/1 beside implement_kind?/1 from one constant, applied
  to RunnerStages.apply/3 (stage reports), session_ended_with/4 (the lease
  reclaim's redrive), reply_story_lock/3 (so only an implement acceptance
  takes the write lock that re-anchors the claim lease) and place/4's resume
  of a ledger id (another kind is dispatch_id_conflict).
- place_review answers a retry of a recorded placement from its row before
  any pre-check, with no new chain entry and no second push unless the
  ledger never recorded the push; another story's id is dispatch_id_conflict.
- A fix may answer only findings of reviews under the story's current claim.
- A review session that reported session_ended answers resends only; a new
  judgement is dispatch_not_accepted.
- session_ended reads the ledger row once (RunnerStages.held_dispatch/3,
  busy-guarded) and routes on its kind; neither path reads it again.
- The ReviewCeilingWorker cron comment states the enqueue and the backstop.

* US-45.3: review round 3 fixes on PR 907

- Ledger session accessors are kind-scoped: accepted_session/4 answers an
  implement row unless the caller names a kind, and the private held/4 must
  be told its kind. RunnerThreadSession.read/5 is implement by default and
  review asks for review. The per-consumer kind checks this made redundant
  are deleted (RunnerStages, RunnerThreads, RunnerReviews, TriageVerdict);
  the session-end writes keep their under-lock kind checks, now tested.
- The review payload is cast before the review row and chain entry exist.
- A resend of a judgement is answered during a custody halt; the halt binds
  new judgements only.
- Freeing a review session's slot never raises in the channel.
- A retry answers from the row only for a sent or accepted dispatch; a
  refused or superseded one is review_dispatch_refused.
- Only a material round-2 finding introduced by a fix opens round 3.
- The loser of a concurrent same-story placement answers from the row and
  never pushes.
- A malformed runner_id or dispatch_id in the body is 422 invalid_uuid.
- The verdict no longer frees the slot; session_ended does. Contract 1.21.0
  text edited in place, v1.json regenerated, digest updated.
- The slot pre-check reads the one runner's row.
- A migration rollback test: down refuses and drops nothing while a review
  exists, and with none drops and restores thread_reviews.
mkreyman added a commit that referenced this pull request Sep 28, 2026
…reviews (#917) (#918)

* US-45.9: an interactive session implements in a change thread and a runner reviews it (#917)

The three rules that keep an interactive claim out of thread mode: review placement
requires a dispatch-made claim, the merge gate reads an unplaced claim as pr, and the
executor fences on a stage row a hand-made story lacks. The reviewer stays a runner, as
Epic 45 decided after #901 and #905.

* Review round 1 on #918: one route derivation and a claimant stage path

The first draft special-cased three rules. Every reader of a claim route derives it from
one query, and the stage row only moves on runner reports, so the story now records an
interactive claim route (mode, base branch, branch) where that query reads, tells
placement and bulk claims apart by an explicit marker, lets the claimant report its own
stage over the runner-reportable edges with head_sha, moves an intake-born row from
queued at claim, starts the CI clock at the ci report, scopes the separation relaxation
to claims with no lineage, requires lease renewal in the tool descriptions, fixes the
separation test so it can run, and lists US-45.1 and US-45.7 as dependencies.
@mkreyman
mkreyman deleted the feature/us-45.3-review-dispatch branch September 29, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant