A PR is one idea, not one knob

[renmengye]

Recent gpt-speedrun PRs show two patterns the old rule produced.

• Legitimate combinations get blocked. The panel blocked a real idea that needed its hyperparameters retuned (Phase 5 core: contained sessions + single-benchmark climb (tsp first) #29: affine RMSNorm with a faster EMA), because the rubric required each component to clear the floor on its own.
• Tuning runs one point at a time. Authors stepped a single knob per attempt without sweeping: Design: the explicit lifecycle of a run #24/Design sketch: planned search, experiment ladder, multi-agent scaling #25 set MLP width to 4×, Phase 4: compute seam, run-state machine, fail-safe tick, chain shim #27 to 6×, Design: environments and containers on Torch #28 raised depth from 14 to 22, each with one launch. A sweep that could run in parallel ran as four sequential gates, and every step made the model bigger, which steps-to-target rewards.

Owner decisions: a combination is fine when the joint effect is genuine and each single effect is documented. A single-point tuning PR may be blocking when the report gives no picture of the landscape (a sweep, neighbouring values, or an ablation); this is a judgment, not an absolute rule. Fundamental ideas are encouraged even when they don't yet beat the best.

What changes (wording and one additive category)

• Panel instruction on research-line targets: a PR is one idea. An idea may bring the few changes it needs, such as the hyperparameters it shifts, when the report documents each change's own effect. Independent ideas bundled together, or a win the report does not attribute, stay blocking.
• Verifier rubric.
  • aggregation now accepts a genuine combination with per-change effects. It still rejects an even blend of small tweaks with no stated mechanism.
  • A new landscape category covers single-point hyperparameter or capacity changes with no sweep, neighbours or ablation behind them. It also asks a size-only win to argue why size is the right lever.
• Author brief.
  • One idea per PR; a larger idea touching several places is welcome with its mechanism stated.
  • Sweep in one array launch and show the landscape around the chosen value.
  • A clear idea that doesn't yet win is a success to report, and the author's research line keeps it.
• Architecture doc: "one hypothesis per PR" becomes "one idea per PR", with attribution required.

Compatibility

Prompts and rubric text only, plus an additive finding category; a reader that does not know landscape treats it as other. No persisted state changes. It applies from the next panel read.

🤖 Generated with Claude Code

[github-actions]

Round 1 — reviewed head c5aff9f4 — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: nothing blocking — 2 advisory notes.

Advisory (non-blocking):

• Suggestion: Research-line brief still requires one clean contribution. [coverage+prose] A PR to main must be one idea, and may include the related changes it needs when the report gives each change’s own effect. (src/outerloop/brief.py:280; high confidence)
• Generated PRs still tell authors to use one hypothesis. [credentials] The changed policy allows one idea with several attributed changes, but generated PR bodies still say one hypothesis per PR, which can discourage the combinations this change permits. (src/outerloop/orchestrator.py:2268; high confidence)

Verdict: two non-blocking policy-alignment changes remain. Rejected: none; the coverage and prose reports are merged because they identify the same stale one-contribution rule in the research-line brief, with prose supplying the required replacement wording.

[renmengye]

Both advisories batched in 86ca856: the research-line brief now says a PR to main is one idea with the few changes it needs, each change's effect in the report; generated PR bodies say one idea per PR. Pinned brief test updated.

[github-actions]

Round 1 — reviewed head 86ca8562 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: no defects found.

[renmengye]

Compatibility statement (added for the 0.3.0rc1 release audit, per RELEASING.md).

• Surfaces: agent-visible instructions (the author brief and the panel rubric), PR-body wording, and a persisted finding category (landscape).
• Oldest state read: v0.2.1 findings without the new category fall back to the existing categories.
• First tick after upgrade (explicit tolerance, owner decision): judges start fresh sessions and use the new rubric immediately. A parked author resumes its existing conversation, which keeps the instructions it started with; the brief is not re-delivered. If the new rubric blocks such a run, the panel's feedback explains the rule and the author can revise. Runs started after the upgrade get the new brief.
• Rollback: safe for records; findings in the new category are read as generic findings by older kernels.
• Fixtures: a run parked under the v0.2.1 brief, resumed and judged with the current code, is added on the release branch for 0.3.0rc1.