Skip to content

ruff_python_spo: callee-name residual detail + the drill's proposer stage - #97

Merged
AdaWorldAPI merged 1 commit into
mainfrom
claude/plain-spo-config-drill
Aug 18, 2026
Merged

ruff_python_spo: callee-name residual detail + the drill's proposer stage#97
AdaWorldAPI merged 1 commit into
mainfrom
claude/plain-spo-config-drill

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

Summary

Increment 2 of the self-adaptive drill loop (#96), scoped to what could actually be measured this round. Two pieces.

1. Callee-name detail on non_literal_assign

x = <call>() residuals used to carry the generic "call" tag. Now, when the callee resolves to a terminal name, detail = "call:<name>" (field(...)"call:field"); an unresolvable callee (a call whose own callee is a call, a lambda, a subscript — never guessed) keeps the generic "call" tag.

Measured payoff on dismech: the 451-row call bucket from #96's histogram decomposes losslessly into three named LinkML schema factories:

355  non_literal_assign/call:PermissibleValue
 96  unresolved_annotation/binop
 91  non_literal_assign/call:EnumDefinition
  5  non_literal_assign/call:ConfigDict

355 + 91 + 5 = 451, exact — this is precisely the drill target #96's own histogram predicted without being able to name.

2. drill.rs — the proposer stage

propose(residuals, min_support) groups a ledger by (reason, detail) for the four reasons dense enough to carry one (curie_constant, non_literal_assign, unresolved_annotation, unresolved_base), thresholded on support. classify_across_corpora(labeled, min_support) runs propose over 2+ labeled corpora and splits each candidate into Generic (cleared the bar in every supplied corpus) vs CorpusScoped (some but not all) — an exact rule, not a similarity heuristic.

Scope boundary — stated in the module doc, and stated again here so it isn't overclaimed: this builds grouping + classification only. It does not build a promotion gate that re-runs extraction with a candidate row "active" and checks the coverage delta — that needs a config-consuming extractor (a trie the plain arm reads before deciding a site's classification) that doesn't exist yet. That's the next increment, not this one.

Measured — and it falsifies my own prediction

Ran plain_propose at min_support=2 over three real corpora (dismech / A2UI SDK / ruff/scripts):

18 candidate rows, 0 generic, 18 corpus_scoped

I'd predicted in the #96 handoff that some row would prove cross-corpus generic. It didn't — worth recording plainly rather than dropping quietly. The closest candidate, unresolved_annotation/binop, fires 96× in dismech and 12× in ruff/scripts but is silent on the A2UI corpus — so under the exact "fires in ALL supplied corpora" rule it stays CorpusScoped. That's the strict rule working as designed: three modest corpora is a small n for "generic," and a fuzzy magnitude-similarity threshold would have manufactured a Generic verdict this data doesn't support. curie_constant rows are, as expected, uniformly CorpusScoped to dismech.

Falsifier discipline

Every new grouping/threshold/classification site verified by a disable run — mutation asserted present, test observed red, restored, full suite green: the min_support filter, the Generic/CorpusScoped split (forced everything to Generic — both cross-corpus tests failed), and the callee-name fallback (both resolved-and-unresolved cases exercised in one fixture).

Test plan


Generated by Claude Code

Two pieces of increment 2, both scoped to what could actually be measured
this round.

1. non_literal_assign's detail now carries the callee name for a resolvable
   call (`field(...)` -> "call:field"), falling back to the generic "call"
   tag when the callee is itself unresolvable (a call, lambda, subscript -
   never guessed). Measured payoff on dismech: the 451-row "call" bucket
   from increment 1 decomposes losslessly into three named LinkML schema
   factories - call:PermissibleValue 355, call:EnumDefinition 91,
   call:ConfigDict 5 (355+91+5=451, exact) - which is exactly the kind of
   drill target increment 1's own histogram predicted without being able
   to name.

2. New drill.rs: the proposer stage. propose() groups a residual ledger by
   (reason, detail) for the four reasons dense enough to carry one, and
   thresholds on support. classify_across_corpora() runs propose() over
   2+ labeled corpora and splits candidates into Generic (cleared the bar
   in EVERY supplied corpus) vs CorpusScoped (some but not all) - an
   exact, falsifiable rule, not a similarity heuristic.

   Scope boundary, stated in the module doc and here: this builds
   grouping + classification ONLY. It does not build a promotion gate
   that re-runs extraction with a candidate row "active" and checks the
   coverage delta - that needs a config-consuming extractor (a trie the
   plain arm reads before deciding a site's classification) that does not
   exist yet. Claiming this stage ratifies rows would overstate what it
   measures.

Measured, min_support=2, three real corpora (dismech / A2UI SDK /
ruff/scripts): 18 candidate rows, ZERO generic. This falsifies my own
prediction from the increment-1 handoff that some row would prove
cross-corpus generic - worth recording plainly rather than quietly
dropping. unresolved_annotation/binop comes closest (96 in dismech, 12 in
ruff/scripts) but is silent on the A2UI corpus, so by the exact "fires in
ALL supplied corpora" rule it stays CorpusScoped. That is the strict rule
working as designed, not a bug: three modest corpora is a small n for
"generic," and the alternative (a fuzzy magnitude-similarity threshold)
would have manufactured a Generic verdict this data does not support.

plain_propose new example: runs classify_across_corpora over N labeled
corpora and prints candidates as TOML - data, not Rust, per the earlier
design note that the proposer must never write code.

Falsifier discipline: every new grouping/threshold/classification site
verified by a disable run (mutation asserted present, test observed red,
restored, full suite green) - the min_support filter, the Generic/
CorpusScoped split, and the callee-name fallback.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AGVLyRZNEKKBSfBDJfbY3V
@cursor

cursor Bot commented Aug 18, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_aba29efd-6479-4211-bcf0-90e524105a81)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review August 18, 2026 05:54
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@AdaWorldAPI
AdaWorldAPI merged commit a91c239 into main Aug 18, 2026
28 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants