Stage 1-r1: the pre-execution design layer, and the HOLD it reports - #835
Merged
Conversation
…s instrumented
Stage 1-r1 replaces the Stage 1 draft that failed adversarial review. This builds the
pre-execution layer the FINAL-PRD asks for -- buildability census machinery, the
task-author firewall, oracle validation, the frozen analysis, the runtime lock, the
power rule and the pilot gate -- and then reports HOLD, because four of the fifteen
acceptance criteria have nothing to be satisfied by yet.
The mechanical screens ran against all four sealed bundles and settled nothing:
repository candidates screen-refuted runner
agent-control-plane 10 0 npm test
agent-operator-score 17 0 npm test
gitseed 22 0 pytest
logic-pro-mcp 13 0 swift test
61 of 62 decision scopes survive intact at the frozen snapshot and the 62nd survives
partially. Every repository has an executable test command. So no candidate is
refutable by machinery, and none is admissible by it either: BUILDABLE asserts that an
oracle exists and discriminates, which is a claim about an artifact.
All 62 rows therefore carry `disposition: null`, and `assertCensusComplete` throws on
the file this commit adds. The same holds for the runtime lock: 17 null fields and
`assertRuntimeLockComplete` throwing on it. The guards are proved against the real
artifacts rather than against fixtures, which is the only proof that they fire where
it matters.
A defect was found in the power arithmetic while writing it. The inverse-normal
transcription dropped a term from the central denominator, so every quantile came out
at about 1/400 of its value and the corpus appeared able to detect 0.1 percentage
points. Fixed, exported, and pinned against published quantiles to five decimals. The
failure mode is worth naming: it flattered the design instead of breaking it.
The corrected arithmetic produced a finding, registered here before any outcome
exists. Over 50 reserve candidates in four fixed strata at 8 repeats per arm, the
detectable difference in Delta is 12.3 to 17.2 percentage points depending on
between-candidate heterogeneity. Ten points needs 12 repeats and homogeneous
candidates. Repeats cannot average heterogeneity away; only more candidates can, and
the corpus is fixed at 62.
Nine guards are registered in the mutation ratchet and the ratchet caught one of them
inert: the pilot gate's effect check was covered by a test that accepted either error
message, and every effect-named key is also an unregistered key, so the assertion
passed with the guard disabled. Binding the test to the effect-specific diagnosis made
it fail under mutation. 32 bound now, up from 22.
Record-Id: r-v5stage1r1designlayer
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: filling the runtime lock with plausible placeholder values | an unpinned runtime that reads as pinned is worse than an empty lock, because the next reader stops asking
Ruled-out: marking the 62 screen-surviving candidates BUILDABLE | the screens can only refute, and calling a candidate buildable without an oracle is the claim the census exists to check
Ruled-out: patching the six defects into the failed Stage 1 draft | its own section 7 says anything but the deferred N makes a change a new preregistration, so amending the document that defines amendment is the failure it guards
Ruled-out: reporting PASS on the eleven satisfied criteria | four unresolved P0/P1 is a HOLD under section 19, and a partial pass reads as readiness to whoever approves execution
Limit: the task-author firewall has still never run. Whoever built this corpus has read all 241 records, so the record-blind half cannot be satisfied from here -- it needs an author whose only inputs are the base tree and the maintenance need, with the manifest proving it
Limit: no oracle exists for any of the 62. Stage 0 recorded that reviewers thought one could be written; between that and a validated discriminating oracle sits the whole of G2, and nothing has crossed it
Limit: the between-candidate variance in the power table is a range I chose to bracket, not a measurement. The pilot supplies the real value, and only then does the detectable effect stop being a family of curves
Verified: 45 new tests pass and 3,641 across the suite; typecheck clean; the mutation ratchet reports 32 bound with no regression. assertCensusComplete and assertRuntimeLockComplete are proved by throwing on the committed artifacts, not on fixtures. The five suite failures under full parallel load -- cdeb-smoke, doctor budget, secret-guard regex cost -- pass when their files run alone
CommitLore — record lintTrailers: clean — 11 commits in Active constraints for the paths this PR touchesLimits (152)
Ruled out (184)
Truncated: 32 lines omitted — the comment hit GitHub's 65000 character limit. Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR. |
…ing that noticed Adding `bench:cdeb:v5:stage1-census` to package.json moved the canonical artifact's source checksum. `package.json` is one of four SOURCE_INPUTS alongside package-lock.json, tsconfig.json and src, so a scripts entry is as much a source change as a function is. The canonical docker build reproduces dist/commitlore.mjs byte-identically -- no src changed -- and only installer/canonical-artifact.json moves. Record-Id: r-v5stage1r1manifest Provenance: authored Certainty: firm Blast: local Undo: easy Verified: `npm run artifact:verify` reports canonical artifact verified e8183a8f; the docker build ran at the pinned node:24-bookworm digest the failure message named, and dist bytes are unchanged
…closed An independent adversarial reviewer read 30 files against a detached worktree of 28699d8, returned HOLD, and found eleven things rather than the four open criteria the validation report already named. Six made concrete claims about behaviour. All six reproduced when run: interval collapse 50 agreeing candidates -> [0.125, 0.125], zero width, superiority declared retry overwrite a success with the same key silently replaced the failure it followed, every assigned key still present wrong repository an observation labelled with a repository it was not assigned to was accepted into that stratum masked degradation a one-candidate repository losing every completion pooled to -1.6 points and passed the -5 margin impossible controls three controls sharing one patch digest, two saying revival=false and one revival=true, validated common-mode drift both arms moved off the freeze together and the arm-versus-arm check passed The interval one is the worst. A cluster bootstrap over candidate point estimates represents between-candidate spread and nothing else, so where candidates agree it reports certainty it does not have. Eight coin flips per arm is not zero uncertainty. It now draws repeats inside each drawn candidate as well. Two findings were design contradictions rather than defects. The power rule required tau^2 -- between-candidate variance in the ON-minus-OFF difference -- while forbidding the sizing step to read any arm comparison, and the 12-candidate pilot cannot estimate a variance regardless. Deferring it left room to substitute a within-arm variance and certify a power the study does not have. It is now frozen at 0.06 in code, and the executable check changed the design: 8 repeats detects 17.2 points against a registered 15-point target, so the envelope is 15 repeats and 1,500 episodes. A 10-point effect is unreachable at any repeat count, because repeats shrink only the binomial term and the corpus is fixed at 62. The other contradiction was quieter. Pilot thresholds required all 12 candidates while the census may legitimately dispose one NOT_BUILDABLE, which puts feasibility pressure behind the decision to call a marginal candidate buildable -- the one decision that has to be free of it. The counted thresholds now follow the buildable subset, no candidate is replaced, and too few is a HOLD. The ratchet earned its place twice. It caught the pilot-gate guard inert, because every effect-named key is also an unregistered key and the test accepted either message. Then it caught the equal-weighting mutation targeting the confidence bound instead of the weighting it names. 41 bound now, from 22 at the start. Record-Id: r-v5stage1r1reviewfixes Provenance: authored Certainty: firm Blast: system Undo: easy Ruled-out: treating the reviewer's findings as claims to weigh | six of them were statements about what the code does, and running the code settled each one in under a minute Ruled-out: keeping 8 repeats and reporting the detectable effect as a range | the range was over a parameter the design had no way to obtain, which is how an operator ends up choosing the favourable end of it Ruled-out: lowering the minimum important effect to what 8 repeats reaches | that is the move the HOLD rule exists to refuse, and writing it into the rule that refuses it would have been circular Ruled-out: claiming the pilot custody finding was closed by the schema | the record now cannot carry an arm contrast and the operator can still have watched the runs; a control over bytes is not a control over people Limit: the pilot-gate custody gap is open and needs role separation -- execution, custody of arm-coded outcomes, and continuation held by different parties. That is an owner decision and it has to be settled before the pilot, not after Limit: the firewall now names who produced the maintenance need and refuses a producer not declared record-blind, but a declaration is not evidence. Making it evidence needs an attestable isolated authoring environment this layer does not have Limit: tau_squared_bound = 0.06 is a frozen assumption, not a measurement. It is conservative in the direction that matters -- lower true heterogeneity means the study detects more than promised -- but nothing here establishes the true value Verified: 57 tests in the r1 file and a clean typecheck; the mutation ratchet reports 41 bound with no regression, having flagged two of my own guards inert before they were fixed; each of the six behavioural findings was reproduced by running the code before the fix and is now a named test; the canonical build reproduces dist byte-identically and artifact:verify reports e8183a8f
…hing was reading `attempt_log_digest` went onto `BuildabilityRow` when NOT_BUILDABLE started requiring the failed construction artifacts. The committed schema is `additionalProperties: false` and did not gain the field, so a row the code is required to emit is a row the schema rejects. Nothing compared them. Now a test validates every committed census row against the committed schema, and every disposition the code can emit against it too. Removing the field from the schema fails that test with `data must NOT have additional properties`, which is the control run before this was committed. `Ajv2020` rather than the default export: the schema declares draft 2020-12, and plain `Ajv` answers `no schema with key or ref "…/2020-12/schema"`. `src/core/schema.ts` already had the idiom, including the `.default` unwrap that ajv-formats needs. Record-Id: r-v5censusschemadrift Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: dropping additionalProperties:false so the schema stops caring | it is the clause that keeps an outcome field off a disposition row, which is the whole reason the census has a schema Verified: 58 tests pass and typecheck is clean; the schema test was run with attempt_log_digest deleted from the schema and failed, then run again with it restored and passed
… the other direction
An end-to-end execution SSOT arrived and governs from here. It answers both questions
this branch had left open for the owner, and it supersedes part of what I registered.
minimum important effect 15pp (my product judgement) -> 20pp (SSOT 9.1)
repeats 15, from inverting an MDE -> 4/5/6 from the
buildable count (9.2)
budget 1,650 episodes -> 400 ceiling
pilot's role in sizing five nuisance parameters -> nothing at all
outcome custody open, flagged to the owner -> role isolation (4)
NOT_BUILDABLE reasons seven of my own naming -> the registered nine (6.1)
The direction matters more than the numbers. I sized the study by solving for the
smallest envelope reaching a target, which lets the target and the budget negotiate
with each other. Section 9 fixes repeats from the buildability census -- frozen before
any episode -- and then reports what that envelope detects. `PERMITTED_PILOT_INPUTS`
is now empty rather than filtered: a channel carrying nothing cannot carry the effect
under a neutral name, and nobody has to be trusted not to look.
Section 9.3's gate is implemented as it is written -- conservative binary simulation --
and run before any episode. It passes:
M=40 rep 4 320 ep power 0.95 M=28 rep 6 336 ep power 0.96
M=36 rep 5 360 ep power 0.97 M=24 rep 6 288 ep power 0.93
M=30 rep 5 300 ep power 0.94
My analytic cross-check disagreed with that and said every branch misses 20pp. The
difference is one term: the simulation as 9.3 names it models binary outcomes and
nothing else, which is the case where delivery helps every candidate equally. Adding
between-candidate variance drops power to 0.90 around tau^2 = 0.02-0.03 and to
0.77-0.86 at 0.06.
I did not substitute my assumption for the registered method, and I did not let the
registered method's optimism go unstated. The gate runs as written and the full
sensitivity table is registered in the same artifact, before any outcome exists, so a
null result has to be read against it.
The ratchet earned its keep a third time: it reported the new artifact-agreement guard
inert, because the test only asserted the current artifacts agree and never that a
drifting one is caught. It now copies the study, adds "9 repeats per arm" to the
preregistration, and requires the throw. 44 bound.
Record-Id: r-v5ssotreconciliation
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: keeping my 15pp/1,650-episode envelope | it was derived by inverting a detectable-effect formula against a variance the pilot was supposed to supply, which is the sizing direction section 9 exists to remove
Ruled-out: running the 9.3 gate with my more conservative variance assumption | that substitutes my judgement for the owner's registered method and would have produced a HOLD the SSOT does not call for
Ruled-out: reporting the 9.3 PASS without the heterogeneity table | the registered simulation assumes candidates benefit equally, and a reader who cannot see what that assumption buys cannot judge a null result
Ruled-out: deleting the analytic sizing path now that it is not the gate | it is the more pessimistic of the two and a disagreement between them is worth seeing rather than averaging
Limit: the 9.3 gate passes under an assumption the study cannot test. Nothing establishes the true between-candidate variance, the 12-candidate pilot cannot estimate one, and the corpus is fixed at 62 so no budget reduces it
Limit: the nine registered reasons are now the closed list, but no candidate has been disposed under any of them -- all 62 remain undecided pending oracle construction
Verified: 60 tests pass and typecheck is clean; assertEnvelopeArtifactsAgree runs against the committed tree and passes, and against a copy with a drifted repeat count and throws; the ratchet reports 44 bound with no regression after flagging one of these guards inert
…ust not see
The task-author firewall needed a sandbox before it needed anything else, and the
obvious construction does not work. A materialized bundle carries the full commit
history and `refs/notes/commitlore`, so an author handed one is a single `git log`
from every record the study is about. The control would be "the author chose not to
look".
So the sandbox is the frozen tree with the git metadata destroyed. Nothing to read
rather than a rule against reading, and the tree digest is over the bytes that
actually went in.
Removing the history does not reach content that is already in the files, and two of
the four repositories carry a record there:
gitseed/docs/adr/ADR-0008-python-floor-widened-to-3.9.md Record-Id: r-gsf501
agent-control-plane/HANDOFF-REPORT.md Record-Id: r-p014live20260814
Both name decisions that are among the 62. A third hit, `gitseed/AGENTS.md`, carries
`Record-Id: r-<6+` and `Ruled-out: <alternative> | <why it lost>` -- format
documentation, not a decision, recorded so the next reader does not re-derive it.
Record-Id matching only finds candidates whose identifier survived into a document,
and the wording is the more common carrier. Comparing every qualified candidate's
`Ruled-out` and `Reason` text against every text file in its own repository, by
shared five-word runs, with the same rulings against a DIFFERENT repository as the
null:
threshold own repository null
>= 1 34 0
>= 3 22 0
>= 5 16 0
>= 20 1 0
The null is zero everywhere, including at one shared run, so a single match against a
candidate's own repository is signal rather than the background rate of English.
34 of 62 are hit.
The firewall consequence is the visible one. The consequence that reaches further is
Stage 0's gate G3 -- "the reason is not obvious from the code" -- which paired blind
reviewers decided and which Stage 0 recorded as having been decided without any
reviewer reading the code. This is the first time the code was read. For a large part
of the corpus the reason's wording is sitting in the tree, which is the condition G3
existed to exclude and had no way to detect, and a SUPPRESSED arm that can read it
does not need delivery to reach it.
Registered now, before any episode, so that a small confirmatory contrast cannot later
be explained by a table that appeared after the result.
Record-Id: r-v5firewallleakscreen
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: giving the task author the materialized bundle | it carries the commit history and the notes ref, so the firewall would rest on the author not running git log
Ruled-out: redacting the leaking files from the sandbox | the base tree is frozen and the episode runs on the real one, so an author working in a different tree authors a task for a repository that does not exist
Ruled-out: reporting the 34 without the cross-repository null | a shared five-word run means nothing until something establishes the rate at which unrelated text shares one, and that rate turns out to be zero
Ruled-out: calling this a global HOLD | a shared run is not a restatement, a hit in docs/adr is read by a different reader than one in the file the task must change, and deciding each case is the census's job
Limit: the screen measures shared wording, not disclosure. A reason that names a function shares wording with the code defining it without saying what was ruled out, and nothing here weighs where the hit landed
Limit: the direction is not established. Record and code were often written together, so wording in common does not say which explains which
Limit: two candidates are refuted outright by Record-Id presence, and the other 32 need per-candidate reading that has not happened. No disposition has been frozen for any of them
Verified: 64 tests pass and typecheck is clean; the sandbox test asserts .git is absent on a real materialization of the sealed agent-operator-score bundle; the leak scan is exercised on a constructed positive, a prose negative and a non-text file; the null control was computed by running the same 62 rulings against a different repository's files and returned 0 at every threshold
…lands on it
The record-blind half of the task-author chain ran end to end for the first time.
Candidate v4-04c1de5e41d66868 in agent-operator-score, in a 354-file tree with the
git metadata destroyed, given the decision's path scope and nothing else.
It returned three maintenance needs. Every file it cited exists. Its needs share
**zero** four-word runs with the candidate's Ruled-out and Reason text:
record embed the canonical reports in specs/doctor-output.v0.json | the ticket
grants fixtures/doctor/*.json, and sibling precedent does not override a
path the ticket names
scout the shared canonical-case table covers only six reports although the
frozen manifest declares eight, leaving generic verdict, required-group
and fail-closed assertions without coverage
The second is the interesting part. The scout independently found the same *area* --
it cites `specs/doctor-output.v0.json`, the file the first Ruled-out line names -- and
an agent fixing that need might well reach for embedding the reports in the spec,
which is exactly what the decision ruled out. That is the shape a usable task has to
have: it must land where the ruled-out approach is a natural and functionally viable
answer, without the task ever having said so.
The prompt is tested for what it does not say. A scout told "there is a ruling here
you must not see" writes around the shape of the thing it was told about, so the
prompt asks for maintenance work and never mentions that a decision exists.
Selection is by external seed rather than by whoever holds the record, and the test
requires the seed to actually move the choice -- a deterministic rule that always
picks the first need is decoration.
Record-Id: r-v5needscoutfirstrun
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: telling the scout a decision exists and to avoid it | it would write around the shape of the thing it was told about, and the avoidance would be the leak
Ruled-out: letting the record-holder pick among the returned needs | at that point the picker has read the ruling and would choose the need running closest to it, which is the selection bias the external seed exists to remove
Ruled-out: restoring .git after codex refused to run without it | the refusal was the sandbox working, and the flag that tells codex the directory is not a repository costs nothing
Limit: one candidate, one scout, one model. FUNCTIONAL-AUTHOR has not run, so no manifest pair exists and the record-blind-before-record-aware ordering is still unproven end to end
Limit: zero four-gram overlap cannot rule out a paraphrase that shares no four-word run. The lexical check is a floor, not a proof of independence
Limit: the encouraging part -- a need landing on the decision boundary -- is one observation and says nothing yet about whether the other 61 behave the same way
Verified: 68 tests pass and typecheck is clean; the run's exit code, event count and output size were checked before the answer was read; every cited path was tested for existence in the sandbox the scout was actually given; the leakage figure comes from detectRecordLeakage over the candidate's real commit-message trailers
…e that has them CI failed on `check` for both Node versions: the sandbox test materializes `agent-operator-score.bundle`, and `.gitignore:15` excludes `bench/cdeb/studies/*/corpus/bundles/`. That exclusion is right -- the bundles mirror private repositories and committing them would publish those repositories -- so the test was the thing that was wrong. It passed locally for the same reason it could never pass anywhere else. The replacement builds its own bundle, and it is a better test than the one it replaces. It creates a repository whose commit message carries `Ruled-out: caching the value globally | it outlives the request` and mirrors the same record into `refs/notes/commitlore`, bundles both refs, and then materializes it. The assertion is not only that `.git` is gone but that `git log` in the sandbox *fails* -- the ruling is unreachable rather than merely not checked out. The old test asserted a history was stripped without ever establishing what was in it. This one puts a known record in twice and shows neither survives. Record-Id: r-v5sandboxtestselfcontained Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: committing the corpus bundles so the test can find them | they are mirrors of private repositories and the gitignore rule exists to keep them out of a public repo Ruled-out: skipping the test when the bundle is absent | it would then be a test that never runs in CI and reports green for having done nothing Verified: 69 tests pass locally and typecheck is clean; the new test builds its own repository, so it depends on nothing outside the checkout; the negative control is the `git log` call, which must throw for the assertion to hold
…refuses the tampered case
FUNCTIONAL-AUTHOR ran in the same tree NEED-SCOUT worked in -- no git metadata, so the
record history and the notes ref were not there to read -- and returned acceptance
criteria for the selected need:
command npm test -w @aos/schema -- doctor-contract
criteria all eight canonical reports the frozen manifest declares are accepted,
each deriving its declared verdict and exit code, all seven required
groups observed, and an invalid report failing closed
The command is grounded rather than plausible: `@aos/schema` is a real workspace with
a real test script in the frozen tree, and the eight fixtures the criteria name are on
disk. The criteria describe observable behaviour and say so -- "they do not require a
particular table, loop, helper, or fixture-loading approach" -- which is what lets two
different correct implementations both pass.
The whole record-blind half now has an artifact at every step:
sandbox .git destroyed, proved by git log throwing
need-scout 3 needs, every cited file real, 0 shared 4-grams with the ruling
task-freezer selection by external seed, tested to move with the seed
functional runnable criteria, 0 shared 4-grams with the ruling
manifest inputs on the allow list, both prose producers declared record-blind
ordering task frozen before oracle passes, and editing the task afterwards
is refused
That last line is the one worth having. An ordering gate that only ever sees the
well-formed case has not been shown to order anything.
What this does not show is in the artifact rather than only here. The oracle half does
not exist; the paired oracle manifest carries a placeholder digest and is there solely
to exercise the ordering check, so G2 is entirely open and this candidate is not
BUILDABLE. The criteria's claim that they fail on the tree today is the author's and
was not executed -- the sandbox has no node_modules and installing them would change
the frozen tree.
Record-Id: r-v5taskchainfirstpair
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: installing node_modules in the sandbox to execute the acceptance command | it changes the frozen tree the digest is taken over, and the tree is the one thing the manifest claims to pin
Ruled-out: recording the oracle manifest as if the oracle existed | it carries a placeholder digest, and a pair that looks complete is worse than an absent one because the next reader stops asking for G2
Ruled-out: letting the ordering gate be tested only on the well-formed pair | a gate that never sees the tampered case has not been shown to order anything
Limit: the oracle half has not been built for any candidate, so gate G2 is untouched and no candidate can be BUILDABLE yet
Limit: zero four-gram overlap is a lexical floor. It cannot rule out a paraphrase that shares no four-word run, and nothing here tests for one
Limit: one candidate, one model, one run of each half. Nothing yet says the other 61 behave the same way, and the leak screen already suggests some will not
Verified: 70 tests pass and typecheck is clean; the acceptance command was resolved against the frozen tree's package.json and workspace manifest rather than taken on trust; both leakage figures come from detectRecordLeakage over the candidate's real commit-message trailers; the ordering negative control was run and refused the edited task
…hat makes it unstudiable
First oracle construction, first frozen disposition. Candidate v4-04c1de5e41d66868 in
agent-operator-score is `NOT_BUILDABLE:no-functionally-passing-violation`, and it was
decided by building the controls and running them rather than by reading the record.
The acceptance suite runs offline with no dependencies at all -- `node --test` on a
repository that declares none -- which settles determinism for this repository.
Good A append the two missing rows to both hardcoded case tables
41 tests, 41 pass compliant, functional PASS
Bad A embed verdict, exit_code and observations into the two
canonical_reports rows of the frozen document
41 tests, 15 pass, 26 fail CONTRACT_ROW_DEAD_FIELD x210
Bad B embed both bodies under a new top-level key in the frozen document
41 tests, 15 pass, 26 fail CONTRACT_ROW_DEAD_FIELD x210
Bad C write the bodies to a sibling specs file
41 tests, 41 pass rejected as a control: nothing
reads the file, so it passes by
being inert
The guard is not a test a patch could edit around. `packages/schema/src/doctor-contract.ts`
carries a closed list of contract top-level fields and a CANONICAL_FIELDS row shape, and
`validateTable` rejects anything beyond them. The product's own validator refuses the
document shape the ruled-out approach requires, so no revival passes acceptance and the
endpoint is unobservable here.
Good A took two attempts and the first failure was informative: the case list is
duplicated in a second table inside `exit-codes-are-derived-from-the-verdict`, which
asserts the two agree. The record-blind FUNCTIONAL-AUTHOR had nonetheless got the
expected values right -- both new reports derive SCORE_BLOCKED with exit 30, which the
fixtures confirm.
The mechanism is worth naming because it predicts more than this candidate. The record
says the ruled-out approach "would have matched all three siblings and passed every
gate", and that was true at the decision moment. Implementing the compliant choice is
what installed the guard that makes it false in the frozen present tree: the same work
that recorded the decision also enforced it. Stage 0's gate G4 asked whether the wrong
path is functionally viable, and reviewers answered from the record's prose -- which
describes the counterfactual past, not the tree the study would run against.
A decision worth recording is often a decision worth guarding, so the best-implemented
decisions may be systematically the least studiable. That is a mechanism, not a
measurement, and n is 1.
Record-Id: r-v5oraclefirstattempt
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: counting Bad C as a functionally passing violation | nothing reads the file it adds, so it passes by being inert, and an unused file is litter rather than a revival
Ruled-out: editing the validator so a revival could pass | that dismantles the guard instead of reviving the approach, and it changes far more than the decision's scope
Ruled-out: concluding from Bad A alone | the record claims the approach would have passed every gate, so one failed attempt is not enough to call the endpoint unobservable
Ruled-out: pinning the census tests to "62 of 62" | they would need editing every time a candidate is disposed, which is how a test stops being read; they now track the census and still fail closed on any open slot
Limit: one candidate of 62. The mechanism predicts that well-implemented decisions are the least usable, and a mechanism that predicts more is not a measurement of more
Limit: the two revivals I built are mine. A more inventive agent might find a shape the validator admits, and the honest reading is that I could not find one, not that none exists
Limit: 61 candidates remain undecided, so nothing here says whether the corpus can clear the SSOT 7.3 floors of 8 buildable per repository and 24 in the reserve
Verified: 71 tests pass and typecheck is clean; every control was executed and its counts are the runner's own output; the disposition carries the attempt log digest and the census summary is regenerated from the rows rather than written beside them
CI failed `guard-mutations`. `stage1-r1-census-refuses-open-disposition` came back inert: its registry entry still named the test as "throws on the census as committed, because 62 dispositions are still open", and I had renamed that test when the census stopped being 62-of-62. Vitest matched nothing, ran nothing, and reported success, so mutating the guard changed no observable outcome. That is the failure this registry exists to catch, arriving in its own machinery. A guard pointing at a renamed test is not a weakened guard, it is an absent one that still appears in the table. The entry now names the current test and the ratchet is back to 44 bound. A static sweep of every registry test_name against its file flagged one more, `literature-lock-requires-complete-evidence`, and that one is a false positive: its names come from a `test.each` with `%s`, so they exist at run time and not in the source text. Which is the reason the ratchet, not a grep, is the check -- it runs the test rather than looking for its name. Record-Id: r-v5guardnamestale Provenance: authored Certainty: firm Blast: local Undo: easy Ruled-out: adding a static check that every registry test_name appears in its file | test.each generates names at run time, so the check reports a false positive on a guard that genuinely binds, and a check that cries wolf gets muted Limit: nothing prevents the next rename from doing this again. The ratchet catches it on the following CI run rather than at the moment of the rename, and that is the guarantee available Verified: reproduced locally before fixing -- the ratchet reported the same inert guard the CI job did; 71 tests pass, typecheck clean, and the ratchet reports 44 bound with no regression after the registry entry was repointed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the Stage 1 draft that failed adversarial review with a distinct r1 preregistration, and builds the pre-execution layer the final PRD specifies. The verdict is HOLD — eleven of the fifteen acceptance criteria are satisfied by evidence, four are not, and under the PRD any unresolved P0/P1 is a hold. No measured episode has been run and none is authorized.
What is here
buildability-v5.tsfirewall-v5.tsoracle-v5.tsruntime-lock-v5.tsanalysis-v5.tseffect-independence-v5.tsstage1-census-v5.tsThe census
The screens ran against all four sealed bundles and refuted nothing. 61 of 62 decision scopes survive intact at the frozen snapshot, the 62nd partially, and every repository has an executable test command. So no candidate is refutable mechanically, and none is admissible mechanically either —
BUILDABLEasserts a validated discriminating oracle, and none has been built.All 62 rows carry
disposition: null, andassertCensusCompletethrows on the committed file. The runtime lock is the same shape: 17 null fields,assertRuntimeLockCompletethrowing on it. The guards are proved against the real artifacts rather than fixtures, which is the only proof they fire where it matters.A defect found while writing the power rule
The inverse-normal transcription dropped a term from the central denominator, so every quantile came out at roughly 1/400 of its value and the corpus appeared able to detect 0.1 percentage points. Fixed, exported, and pinned against published quantiles to five decimals. The failure mode flattered the design rather than breaking it.
What the corrected arithmetic says
Registered before any outcome exists: over 50 reserve candidates in four fixed strata at 8 repeats per arm, the detectable difference in Delta is 12.3 to 17.2 percentage points depending on between-candidate heterogeneity. Ten points needs 12 repeats and homogeneous candidates. Repeats cannot average heterogeneity away; only more candidates can, and the corpus is fixed at 62.
A null result from this study is therefore not evidence of no effect, and the preregistration says so.
Mutation ratchet
Nine new guards registered. The ratchet caught one of them inert on the first run — the pilot gate's effect check was covered by a test that accepted either error message, and every effect-named key is also an unregistered key, so the assertion passed with the guard disabled. Binding the test to the effect-specific diagnosis made it fail under mutation. 32 bound, up from 22.
What would close the HOLD
The firewall cannot be satisfied from the context that built the corpus: whoever did has read all 241 records. It needs a record-blind author.
Verification
45 new tests, 3,641 across the suite; typecheck clean; ratchet reports 32 bound with no regression. Five suite failures under full parallel load —
cdeb-smoke, doctor budget,secret-guardregex cost — pass when their files run alone.