Skip to content

ruff_python_spo: the config-consuming extractor and the promotion gate - #98

Merged
AdaWorldAPI merged 1 commit into
mainfrom
claude/plain-spo-optional-unwrap-gate
Aug 18, 2026
Merged

ruff_python_spo: the config-consuming extractor and the promotion gate#98
AdaWorldAPI merged 1 commit into
mainfrom
claude/plain-spo-optional-unwrap-gate

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

Summary

Increment 3 of the self-adaptive drill loop, closing the boundary drill.rs's module doc has named twice now: "this does NOT build a promotion gate." One rule wired end-to-end — not all four residual reasons at once. A real config surface plus a real gate is worth more than a wide, partially-wired one.

PlainDrillConfig::unwrap_optional_annotation resolves T | None / None | T to T's own field_type reading. Chosen because it was the only cross-corpus candidate from #97's measurement (dismech 96, ruff/scripts 12, genuinely absent — not merely below threshold — on the A2UI SDK), unlike the LinkML call:PermissibleValue/call:EnumDefinition factories, which are corpus-scoped and would need a per-corpus row, not a generic one. field_type_from_annotation recurses into the non-None operand; a chained union (str | int | None) has a BinOp as its left operand, so the shape check fails and it stays unresolved rather than guessing which arm to keep. Off by defaultPlainDrillConfig::default() is pinned byte-identical to the config-free path.

The gate

drill::ratify_optional_unwrap runs extraction with the rule OFF and ON over one source, then independently verifies the claim two ways that share no code with the resolver:

  1. A separately-written shape test (is_shaped_t_or_none), applied by re-parsing the source and re-walking the raw AST directly — not by trusting field_type_from_annotation's own success/failure — counting how many of the newly-resolved sites are genuinely T | None shaped. Must equal the resolver's own count exactly, in either direction: the resolver claiming to fix something the independent check disputes is exactly as much a defect as missing one it should have caught.
  2. Every resolved value is checked against a real-type-name shape (non-empty, alphanumeric) rather than trusted as "present, so fine."

Measured, real corpora, per-file (plain_ratify example)

dismech (84 files):      baseline=96 resolved=91 verified=91 ratified=YES
ruff/scripts (23 files): baseline=12 resolved=12 verified=12 ratified=YES
a2ui sdk (127 files):    baseline=0  resolved=0  verified=0  ratified=YES

dismech's 96 − 91 = 5 gap is the honest remainder: non-optional binop shapes (chained unions, int | str) the rule correctly declines rather than guesses on — the ratification proves the claim is exact, not that the rule resolves everything.

A falsifier finding worth recording rather than discarding

The first disable run on the independent verifier (is_shaped_t_or_none forced to always return true) reported the target test green under mutation — genuinely uninformative, not a false pass: the independent-verification loop only visits sites the resolver itself flipped, and a correct resolver structurally can never disagree with a correct independent check without a second bug existing first. Traced to the right level: RatificationReport::ratified()'s own disagreement-detection logic now has direct unit coverage (ratified_is_false_when_the_counts_disagree_in_either_direction, four hand-built cases), disable-run verified on that function directly (forced to true, the new test failed; restored, green). Two more disable runs on the resolver itself (optional_operand neutered; the config gate bypassed) both verified red-then-green normally.

A process note, for the record

Mid-build I reused the branch name from #97 (claude/plain-spo-config-drill) for this increment's git checkout -b; since that branch already existed (merged), the checkout failed silently and every edit landed on local main instead. Caught before push — origin/main was never touched — by noticing uvx prek run --from-ref main --to-ref HEAD reported "no files to check" on a branch with real changes. Moved the commit to this correctly-named branch, reset local main back to origin/main, and re-ran every gate on the corrected branch before this push.

Test plan

  • cargo test -p ruff_python_spo84/84 (75 pre-existing + 9 new)
  • cargo clippy -p ruff_python_spo --all-targets -- -D warnings — clean
  • cargo fmt + RUSTDOCFLAGS="-D warnings" cargo doc --no-deps -p ruff_python_spo — clean (two intra-doc links de-linked, same pattern as prior PRs)
  • uvx prek run --from-ref main --to-ref HEAD — all hooks pass, tree clean after
  • plain_ratify output reproduced over all three corpora, pasted above

Generated by Claude Code

…nables

Increment 3 of the self-adaptive drill loop, closing the boundary drill.rs
named twice now ("this does NOT build a promotion gate"). One rule wired
end-to-end, not all four residual reasons at once - a real config surface
plus a real gate is worth more than a wide, partially-wired one.

PlainDrillConfig::unwrap_optional_annotation resolves `T | None` / `None
| T` to T's own field_type reading. Chosen because it was the only
cross-corpus candidate from #97's measurement (dismech 96, ruff/scripts
12, genuinely absent - not merely below threshold - on the A2UI SDK),
unlike the LinkML call:PermissibleValue/call:EnumDefinition factories,
which are corpus-scoped and would need a per-corpus row, not a generic
one. field_type_from_annotation recurses into the non-None operand;
a chained union (`str | int | None`) has a BinOp as its left operand, so
the shape check fails and it stays unresolved rather than guessing which
arm to keep. Off by default - PlainDrillConfig::default() is pinned
byte-identical to the config-free path.

drill::ratify_optional_unwrap is the gate: runs extraction with the rule
OFF and ON over one source, then independently verifies the claim two
ways that share no code with the resolver -
  1. a SEPARATELY-WRITTEN shape test (is_shaped_t_or_none), applied by
     re-parsing the source and re-walking the raw AST directly - not by
     trusting field_type_from_annotation's own success/failure - counting
     how many of the newly-resolved sites are genuinely T|None shaped.
     Must equal the resolver's own count exactly, in EITHER direction:
     the resolver claiming to fix something the independent check
     disputes is exactly as much a defect as missing one it should have
     caught.
  2. every resolved value is checked against a real-type-name shape
     (non-empty, alphanumeric) rather than trusted as "present, so fine".

Measured, real corpora, per-file (`plain_ratify` example):
  dismech (84 files):      baseline=96 resolved=91 verified=91 ratified=YES
  ruff/scripts (23 files): baseline=12 resolved=12 verified=12 ratified=YES
  a2ui sdk (127 files):    baseline=0  resolved=0  verified=0  ratified=YES

dismech's 96-91=5 gap is the honest remainder: non-optional binop shapes
(chained unions, `int | str`) the rule correctly declines rather than
guesses on - the ratification proves the claim is EXACT, not that the
rule resolves everything.

Falsifier discipline, including a finding worth recording rather than
discarding: the first disable run on the independent verifier
(is_shaped_t_or_none forced to always return true) reported the target
test green under mutation - genuinely uninformative, not a false pass,
because the independent-verification loop only visits sites the resolver
itself flipped, and a correct resolver structurally can never disagree
with a correct independent check without a SECOND bug existing first.
Traced to the right level: RatificationReport::ratified()'s own
disagreement-detection logic now has direct unit coverage
(ratified_is_false_when_the_counts_disagree_in_either_direction, four
hand-built cases), disable-run verified on THAT function directly
(forced to `true`, the new test failed; restored, green). Two more
disable runs on the resolver itself (optional_operand neutered; the
config gate bypassed) both verified red-then-green normally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AGVLyRZNEKKBSfBDJfbY3V
@cursor

cursor Bot commented Aug 18, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_73a1f384-580a-482c-bafe-55924ecd46d2)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review August 18, 2026 06:21
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@AdaWorldAPI
AdaWorldAPI merged commit fbee873 into main Aug 18, 2026
29 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants