fix(scoring): exclude operational constants from unmodeled scoring drift - #1044
Conversation
|
Tip 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 ✅ Gittensory review — safe to merge
✅ Approved — safe to merge Review summary Suggestions
Risks
Nits — 1 non-blocking
Review context
Contributor next steps
Signal definitions
Review detailsGenerated from public PR metadata and the diff. Advisory only; deterministic signals remain authoritative. The changes correctly introduce a filter to exclude known operational constants from unmodeled scoring drift detection and update tests accordingly. No build-breaking issues or runtime errors are introduced. Suggestions
Risks
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
|
🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 🛠️ Maintainer review — Held for maintainer reviewReviewed 3 changed file(s) — two independent AI reviewers, synthesized. 🛠️ Request changes. (reviewers split: request changes / merge) Address the suggestions below before merging.
Merge readiness: ✅ Mergeable · all CI checks green/neutral/skipped. Summary The change introduces a duplicate declaration of 🛑 Concerns raised — review before merging (reviewers were not unanimous)
💡 Nits — non-blocking, optional
|
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1044 +/- ##
=======================================
Coverage 94.88% 94.88%
=======================================
Files 151 151
Lines 18271 18272 +1
Branches 6615 6615
=======================================
+ Hits 17337 17338 +1
Misses 419 419
Partials 515 515 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
b5c211d to
f7b4f8c
Compare
Refs JSONbored#809 Co-authored-by: Cursor <cursoragent@cursor.com>
Exercise the unmodeled.length > 12 ellipsis path in refreshScoringModelSnapshot so project coverage does not regress after operational constants are filtered. Co-authored-by: Cursor <cursoragent@cursor.com>
be11fb7 to
6ab9989
Compare
ghost
left a comment
There was a problem hiding this comment.
Gittensory approves — the gate is satisfied and CI is green.
Refs #809
Summary
findUnmodeledConstantKeystreated every upstream numeric constant outsideSCORING_CONSTANT_NAMESas "unmodeled scoring drift." That included operational/infra constants the scorer never consumes — HTTP timeouts, byte limits, UID sentinels, subprocess budgets, etc.Those false positives inflated snapshot warnings and could trigger high-severity upstream drift reports without any actual scoring gap.
Problem
Upstream
constants.pyincludes both scoring dimensions and operational knobs. Examples that were incorrectly flagged:SECONDS_PER_DAY,SECONDS_PER_HOURGITHUB_HTTP_TIMEOUT_SECONDS,MIRROR_HTTP_TIMEOUT_SECONDS,MIRROR_MAX_ATTEMPTSTREE_SITTER_PARSE_TIMEOUT_MICROS,SCORING_SUBPROCESS_BUDGET_SMAX_FILE_SIZE_BYTESRECYCLE_UID,ISSUES_TREASURY_UID,MAX_ISSUE_IDA genuinely unmodeled scoring constant like
NOVELTY_BONUS_SCALARshould still surface.Changes
src/scoring/model.ts— addNON_SCORING_UPSTREAM_CONSTANT_NAMESignore-set; filter operational constants out offindUnmodeledConstantKeys.test/unit/scoring.test.ts— regression test: operational-only slice →[]; operational +NOVELTY_BONUS_SCALAR→ only the scoring gap.Detection-only change — no score preview math changes.
Scope boundary
Does not model
MAX_LINES_SCORED_FOR_NON_CODE_EXTor other dropped scoring constants (#809 / #991 follow-up). Those correctly remain flagged as genuinely unmodeled after this noise reduction.API / OpenAPI / MCP contract
No schema changes. Fewer false positives in snapshot warnings and upstream drift reports.
Migration / deploy / secrets
None.
Security / privacy
None.
Validation
All gates passed locally on branch
fix/exclude-operational-constants-from-drift.Distinct from prior PRs
Unrelated to label multiplier selection (#994) or fractional underscore parsing (#992).