Skip to content

Pre-mortem P0/P1: noul polarity, aggregate floor, ledger swallow, eval/prod gap #1

Description

@webdevtodayjason

Hand-off for Claude Code / Grok

Full write-up (file-level, prioritized):

→ docs/PREMORTEM.md

Also indexed in docs/README.md.

TL;DR — fix P0 before wiring agents on ask().action

Until then prefer completionCheck().verdict (+ result_is_echo), not ask().action.

P0

  1. Noul polarity — confidenceOf in src/policy.ts uses Math.max(noul, 1-noul), so confident false bands as accept.
  2. Aggregate lie — all_parts_at_least_0.7 parses the floor then ignores it (resolveAggregate → plain minConfidence).
  3. Silent ledger loss — fileLedger.append empty catch; partial JSONL on mid-flight failure.
  4. Echo beside verdict — hosts checking only verdict === "done" can ship restatements.

P1

  • Live timeout_ms: 750 vs eval 30s; judge no-retry vs eval 5×; 30/30 not in npm test; README MIT vs package.json UNLICENSED.

Suggested patch order

See bottom of docs/PREMORTEM.md.


Reviewed 2026-09-21 against src/, policy.json, tests, DESIGN/USAGE/README claims.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions