Measure the chain in real use, and revisit its defaults

[PierreMardon]

Status

Open, not done: a way to start without waiting for real use is proposed. On 2026-10-08 the developer proposed to clone a public project with a strong harness, to be selected, and to play real needs there, a whole campaign on the prompts as they stand among them: see the comment of that day. Read again on 2026-10-05: the defaults named here still stand, max_autonomous_passes is 3, and since #40 the same ceiling bounds the reworks in planning; surface-status gate runs the project's full gates; every slice, review and fix gets a fresh agent.

What to measure

The chain's defaults were set before any real use: three autonomous passes, full gates run by the script at every pass, a fresh agent for every slice, review and fix. Once projects use it, measure per plan:

• the passes used, and the stops by cause (conformant, ceiling, break, alarm, refusal);
• the review findings by class (defect, deviation, break), and the breaks the developer accepted or refused;
• the gate runs, their duration and their share of the loop's time;
• the cost and the duration of each command.

Most of it can be read from the journals and reports the plans leave on the main branch; a surface-status subcommand could aggregate them.

Then revisit the defaults with the numbers: the passes, the gate runs at every pass (their commands are found at planning since #18), the fresh agent per step.

[PierreMardon]

Proposed by the developer on 2026-10-08: a clone of a public project with a strong harness

Two proposals, taken together:

• Not to wait for real use. Clone a public project whose gates are strong, to be selected, install the chain in it, and play real needs there.
• Play there the whole campaign that main has not had. The prompts as they stand were measured in parts only: the extractor of A blueprint whose long sections hold a title in bold on each thing the developer comes back for #111 on plans drawn again, the hand over of A hand over that tells none of the steps that led to it, and gives the address of the pull request #113 on two runs stopped at the hand over, and the end to end scenarios on the prompts of the day before.

What it would give, and what to settle (the session's reading, not the developer's words)

It answers what the synthetic host cannot: code of a real size, conventions nobody wrote for the chain, and gates that take real time, which is what this issue wants measured, the share of the gate runs in the loop, the passes and the stops by cause, the findings by class.

• The needs. A project's own history gives them, as the corpus builds its cases: an issue closed by a merged pull request is a need, the tests that pull request added are acceptance tests kept from the agents, and its diff is a reference. The clone is taken at the commit before.
• The project. Gates that run in one command, in minutes, with no network and no secret; conventions written down; a history with small features that each came with tests. A project a model knows by heart would be played from memory: a recent change, or a project little known, guards against it.
• The developer. The evaluations play them with a model that answers from a written brief: the brief is to write from the issue and its discussion.
• What it compares with. Nothing yet: a first campaign there is a baseline. The corpus stays the campaign that compares with the five kept, about 31 USD for the six cases twice.

Nothing is selected and nothing is played.

Status

Open, not done: a way to start without waiting for real use is proposed.