ruff_python_spo: the config-consuming extractor and the promotion gate - #98
Merged
Merged
Conversation
…nables
Increment 3 of the self-adaptive drill loop, closing the boundary drill.rs
named twice now ("this does NOT build a promotion gate"). One rule wired
end-to-end, not all four residual reasons at once - a real config surface
plus a real gate is worth more than a wide, partially-wired one.
PlainDrillConfig::unwrap_optional_annotation resolves `T | None` / `None
| T` to T's own field_type reading. Chosen because it was the only
cross-corpus candidate from #97's measurement (dismech 96, ruff/scripts
12, genuinely absent - not merely below threshold - on the A2UI SDK),
unlike the LinkML call:PermissibleValue/call:EnumDefinition factories,
which are corpus-scoped and would need a per-corpus row, not a generic
one. field_type_from_annotation recurses into the non-None operand;
a chained union (`str | int | None`) has a BinOp as its left operand, so
the shape check fails and it stays unresolved rather than guessing which
arm to keep. Off by default - PlainDrillConfig::default() is pinned
byte-identical to the config-free path.
drill::ratify_optional_unwrap is the gate: runs extraction with the rule
OFF and ON over one source, then independently verifies the claim two
ways that share no code with the resolver -
1. a SEPARATELY-WRITTEN shape test (is_shaped_t_or_none), applied by
re-parsing the source and re-walking the raw AST directly - not by
trusting field_type_from_annotation's own success/failure - counting
how many of the newly-resolved sites are genuinely T|None shaped.
Must equal the resolver's own count exactly, in EITHER direction:
the resolver claiming to fix something the independent check
disputes is exactly as much a defect as missing one it should have
caught.
2. every resolved value is checked against a real-type-name shape
(non-empty, alphanumeric) rather than trusted as "present, so fine".
Measured, real corpora, per-file (`plain_ratify` example):
dismech (84 files): baseline=96 resolved=91 verified=91 ratified=YES
ruff/scripts (23 files): baseline=12 resolved=12 verified=12 ratified=YES
a2ui sdk (127 files): baseline=0 resolved=0 verified=0 ratified=YES
dismech's 96-91=5 gap is the honest remainder: non-optional binop shapes
(chained unions, `int | str`) the rule correctly declines rather than
guesses on - the ratification proves the claim is EXACT, not that the
rule resolves everything.
Falsifier discipline, including a finding worth recording rather than
discarding: the first disable run on the independent verifier
(is_shaped_t_or_none forced to always return true) reported the target
test green under mutation - genuinely uninformative, not a false pass,
because the independent-verification loop only visits sites the resolver
itself flipped, and a correct resolver structurally can never disagree
with a correct independent check without a SECOND bug existing first.
Traced to the right level: RatificationReport::ratified()'s own
disagreement-detection logic now has direct unit coverage
(ratified_is_false_when_the_counts_disagree_in_either_direction, four
hand-built cases), disable-run verified on THAT function directly
(forced to `true`, the new test failed; restored, green). Two more
disable runs on the resolver itself (optional_operand neutered; the
config gate bypassed) both verified red-then-green normally.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AGVLyRZNEKKBSfBDJfbY3V
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_73a1f384-580a-482c-bafe-55924ecd46d2) |
AdaWorldAPI
marked this pull request as ready for review
August 18, 2026 06:21
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Merged
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Increment 3 of the self-adaptive drill loop, closing the boundary
drill.rs's module doc has named twice now: "this does NOT build a promotion gate." One rule wired end-to-end — not all four residual reasons at once. A real config surface plus a real gate is worth more than a wide, partially-wired one.PlainDrillConfig::unwrap_optional_annotationresolvesT | None/None | TtoT's ownfield_typereading. Chosen because it was the only cross-corpus candidate from #97's measurement (dismech 96, ruff/scripts 12, genuinely absent — not merely below threshold — on the A2UI SDK), unlike the LinkMLcall:PermissibleValue/call:EnumDefinitionfactories, which are corpus-scoped and would need a per-corpus row, not a generic one.field_type_from_annotationrecurses into the non-Noneoperand; a chained union (str | int | None) has aBinOpas its left operand, so the shape check fails and it stays unresolved rather than guessing which arm to keep. Off by default —PlainDrillConfig::default()is pinned byte-identical to the config-free path.The gate
drill::ratify_optional_unwrapruns extraction with the rule OFF and ON over one source, then independently verifies the claim two ways that share no code with the resolver:is_shaped_t_or_none), applied by re-parsing the source and re-walking the raw AST directly — not by trustingfield_type_from_annotation's own success/failure — counting how many of the newly-resolved sites are genuinelyT | Noneshaped. Must equal the resolver's own count exactly, in either direction: the resolver claiming to fix something the independent check disputes is exactly as much a defect as missing one it should have caught.Measured, real corpora, per-file (
plain_ratifyexample)dismech's
96 − 91 = 5gap is the honest remainder: non-optionalbinopshapes (chained unions,int | str) the rule correctly declines rather than guesses on — the ratification proves the claim is exact, not that the rule resolves everything.A falsifier finding worth recording rather than discarding
The first disable run on the independent verifier (
is_shaped_t_or_noneforced to always returntrue) reported the target test green under mutation — genuinely uninformative, not a false pass: the independent-verification loop only visits sites the resolver itself flipped, and a correct resolver structurally can never disagree with a correct independent check without a second bug existing first. Traced to the right level:RatificationReport::ratified()'s own disagreement-detection logic now has direct unit coverage (ratified_is_false_when_the_counts_disagree_in_either_direction, four hand-built cases), disable-run verified on that function directly (forced totrue, the new test failed; restored, green). Two more disable runs on the resolver itself (optional_operandneutered; the config gate bypassed) both verified red-then-green normally.A process note, for the record
Mid-build I reused the branch name from #97 (
claude/plain-spo-config-drill) for this increment'sgit checkout -b; since that branch already existed (merged), the checkout failed silently and every edit landed on localmaininstead. Caught before push —origin/mainwas never touched — by noticinguvx prek run --from-ref main --to-ref HEADreported "no files to check" on a branch with real changes. Moved the commit to this correctly-named branch, reset localmainback toorigin/main, and re-ran every gate on the corrected branch before this push.Test plan
cargo test -p ruff_python_spo— 84/84 (75 pre-existing + 9 new)cargo clippy -p ruff_python_spo --all-targets -- -D warnings— cleancargo fmt+RUSTDOCFLAGS="-D warnings" cargo doc --no-deps -p ruff_python_spo— clean (two intra-doc links de-linked, same pattern as prior PRs)uvx prek run --from-ref main --to-ref HEAD— all hooks pass, tree clean afterplain_ratifyoutput reproduced over all three corpora, pasted aboveGenerated by Claude Code